-- Logs begin at Thu 2022-07-21 19:47:05 UTC, end at Thu 2022-07-21 20:49:40 UTC. -- Jul 21 20:24:35.732778 dv-cinder-820027-13-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 21 20:24:35.743202 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 21 20:24:35.745249 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 21 20:24:35 2022] *** Jul 21 20:24:35.745249 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 21 20:24:35.745249 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 21 20:24:35.745249 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: nodename: dv-cinder-820027-13-iscsi Jul 21 20:24:35.745249 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: machine: x86_64 Jul 21 20:24:35.745249 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: clock source: unix Jul 21 20:24:35.745249 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: pcre jit disabled Jul 21 20:24:35.745249 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: detected number of CPU cores: 4 Jul 21 20:24:35.745249 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: current working directory: / Jul 21 20:24:35.745249 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: detected binary path: /usr/bin/uwsgi-core Jul 21 20:24:35.745854 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: your processes number limit is 47680 Jul 21 20:24:35.745854 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: your memory page size is 4096 bytes Jul 21 20:24:35.745854 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: detected max file descriptor number: 2048 Jul 21 20:24:35.745854 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: lock engine: pthread robust mutexes Jul 21 20:24:35.745854 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: thunder lock: enabled Jul 21 20:24:35.745854 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 21 20:24:35.745854 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 21 20:24:35.788963 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: Python main interpreter initialized at 0x563d841729c0 Jul 21 20:24:35.788963 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: python threads support enabled Jul 21 20:24:35.788963 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: your server socket listen backlog is limited to 100 connections Jul 21 20:24:35.788963 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: your mercy for graceful operations on workers is 90 seconds Jul 21 20:24:35.789338 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: mapped 403077 bytes (393 KB) for 2 cores Jul 21 20:24:35.789415 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: *** Operational MODE: preforking *** Jul 21 20:24:35.789668 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: *** uWSGI is running in multiple interpreter mode *** Jul 21 20:24:35.789668 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: spawned uWSGI master process (pid: 103654) Jul 21 20:24:35.789862 dv-cinder-820027-13-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 21 20:24:35.791213 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: spawned uWSGI worker 1 (pid: 103663, cores: 1) Jul 21 20:24:35.791213 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: spawned uWSGI worker 2 (pid: 103664, cores: 1) Jul 21 20:24:35.791213 dv-cinder-820027-13-iscsi devstack@keystone.service[103654]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 21 20:24:41.308814 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 20:24:41.308814 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: from cryptography.utils import int_from_bytes Jul 21 20:24:41.309662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 20:24:41.309662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: from cryptography.utils import int_from_bytes Jul 21 20:24:41.310357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 20:24:41.310357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: from cryptography.utils import int_from_bytes Jul 21 20:24:41.310814 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 20:24:41.310814 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: from cryptography.utils import int_from_bytes Jul 21 20:24:41.359814 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 20:24:41.360016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 20:24:41.360191 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] command line args: [] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 20:24:41.360464 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 20:24:41.360641 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ================================================================================ {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 20:24:41.360728 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 20:24:41.360894 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] admin_token = **** {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.360976 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 20:24:41.361170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] command line args: [] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 20:24:41.361272 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] config_dir = [] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.361353 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.361465 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 20:24:41.361557 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] config_source = [] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.361668 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ================================================================================ {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 20:24:41.361748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] debug = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.361870 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.362154 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] admin_token = **** {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.362267 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.362370 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] config_dir = [] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.362448 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] insecure_debug = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.362601 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.362687 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.362821 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] config_source = [] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.362936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.363040 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] debug = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.363144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] list_limit = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.363253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.363386 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] log_config_append = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.363491 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.363592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.363700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] log_dir = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.363797 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] insecure_debug = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.363899 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] log_file = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.364002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.364103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.364253 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.364357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.364502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.364611 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] list_limit = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.364699 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] log_config_append = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.364822 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.364908 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.364988 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.365153 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] log_dir = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.365233 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.365361 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] log_file = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.365439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.365561 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.365643 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.365728 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.365802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.365917 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.366033 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.366147 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.366233 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.366347 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.366473 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.366551 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.366688 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.366776 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.366937 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] notification_format = cadf {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.367015 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.367187 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.367274 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.367422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.367656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.367774 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.71/identity {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.367900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.368016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] publish_errors = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.368131 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.368251 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.368365 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] notification_format = cadf {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.368487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.368609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.368729 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.368851 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.369062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.369198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.71/identity {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.369327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] publish_errors = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.369452 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] standard_threads = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.369574 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.369754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] strict_password_check = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.369880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.369996 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.370116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.370233 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] use_eventlog = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.370355 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.370470 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] use_journal = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.370585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.370698 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] use_json = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.370698 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] use_stderr = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.370911 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] standard_threads = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.371038 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] use_syslog = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.371153 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] strict_password_check = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.371269 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] watch_log_file = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.371383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.371383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] use_eventlog = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.371597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.371720 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] use_journal = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.371836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.371956 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] use_json = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.371956 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] use_stderr = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.372185 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.372302 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] use_syslog = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.372447 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.372573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] watch_log_file = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:24:41.372695 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.372811 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.372927 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.373041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.373186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.373314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.373431 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.373551 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.373551 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.373781 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.373905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.374022 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.374143 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.374265 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.374387 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.374503 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.374697 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.374816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.374920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.375033 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.375146 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.375275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.375401 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.375521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.375636 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.375750 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.375870 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.375870 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.376094 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.376209 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.376323 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.376438 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.376551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.376673 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.376794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.376909 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.377023 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.377189 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.377313 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.377437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.377554 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.377675 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.377790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.377904 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.378025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.378149 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.378264 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.378378 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.378491 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.378612 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.378727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.378850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.378972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.379093 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.379214 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.379314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.379314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.379513 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.379613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.379710 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.379812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.379911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.380007 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.380182 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.380284 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.380284 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.380463 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.380559 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.380662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.380772 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.380870 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.380972 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.381104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.381209 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.381307 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.381422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.381539 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.381661 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.381767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.381868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.381981 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.382104 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.382220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.382342 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.382447 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.382555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.382670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.382794 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.382910 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.383029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.383141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.383261 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.383377 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.383499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.383605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.383713 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.383827 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.383941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.384091 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.384220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.384337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.384452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.384557 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.384659 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.384770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.384886 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.385002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.385220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.385338 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.385444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.385560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.385676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.385811 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.385920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.386046 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.386164 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.386289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.386406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.386531 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.386654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.386771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.386895 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.387023 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.387139 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.387264 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.387381 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.387497 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.387613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.387729 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.387844 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.387973 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.388599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.388717 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.388833 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.388950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.389081 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.389213 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.389332 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.389452 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.389568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.389685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.389801 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.389926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.390045 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.390162 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.390278 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.390394 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.enabled = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.390510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.enabled = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.390636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.390754 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.390871 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.390987 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.391105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.391221 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.391350 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.391474 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.391592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.391708 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.391825 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.391954 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.392159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.392288 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.392406 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.392527 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.392738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.392858 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.392984 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.393174 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.393295 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.393411 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.393538 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.393657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.393783 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.393900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.394016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.394131 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.394248 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.394373 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.394499 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.394625 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.394743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.394865 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.394982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.395098 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.395220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.395348 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.395475 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.395592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.395708 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.395833 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.395951 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.396079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.396197 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.396314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.396431 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.396547 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.396664 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.396788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.396906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.397021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.397150 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.397268 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.397385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.397513 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.397631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] auth.external = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.397747 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] auth.external = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.397863 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] auth.mapped = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.397985 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] auth.mapped = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.398103 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.398255 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.398386 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.398519 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.398638 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] auth.password = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.398755 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] auth.password = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.398871 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] auth.token = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.398987 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] auth.token = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.399103 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.399228 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.399346 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] catalog.caching = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.399468 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] catalog.caching = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.399585 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.399701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.399817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.399942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.400060 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.400177 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.400294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.400419 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.400545 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.400671 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.400790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] credential.caching = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.400907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] credential.caching = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.401023 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] credential.driver = sql {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.401182 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] credential.driver = sql {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.401301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.401430 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.401550 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.401668 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.401785 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.401902 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.402028 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.402146 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.402264 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.402380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.402497 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.402614 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.402740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.402858 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.402976 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.403093 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.403210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.403326 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.403451 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.403570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.403742 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.404377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.404503 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.404622 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.404755 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.404874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.404999 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.405132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.405252 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] federation.caching = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.405369 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] federation.caching = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.405499 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.405619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.405737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] federation.driver = sql {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.405855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] federation.driver = sql {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.405972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.406095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.406225 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.406352 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.406470 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.406588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.406706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.406823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.407019 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.407140 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.407258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.407374 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.407491 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.407607 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.407734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.407852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.407977 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.408094 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.408219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] identity.caching = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.408338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] identity.caching = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.408465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.408584 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.408701 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.408818 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.408937 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.409054 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.409245 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.409364 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.409487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] identity.driver = sql {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.409610 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] identity.driver = sql {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.409728 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.409860 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.409988 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.410106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.410223 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.410340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.410458 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.410458 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.410686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.410805 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.410922 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.411040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.411157 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.411274 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.411400 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.411518 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.411635 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.411758 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.411860 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.411984 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.412104 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.412222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.412340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.412340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.412559 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.412684 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.412804 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.412920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.413037 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.413177 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.413295 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.413429 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.413550 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.413667 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.413784 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.413900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.414017 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.414142 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.414262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.414380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.414497 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.414613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.414731 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.414859 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.414979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.415094 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.415212 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.415329 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.415445 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.415570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.415690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.415806 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.415923 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.416040 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.416157 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.416282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.416401 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.416518 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.416635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.416752 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.416868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.password = **** {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.416993 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.password = **** {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.417124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.417243 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.417372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.417490 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.417607 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.417732 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.417851 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.417968 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.418084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.418201 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.418318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.418444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.418562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.418679 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.418796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.418913 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.419030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.419155 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.419303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.419432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.419550 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.419674 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.419793 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.419918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.420038 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.420155 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.420273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.420389 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.420507 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.420636 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.420756 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.420872 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.420995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.421203 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.421324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.421455 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.421577 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.421703 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.421823 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.421939 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.422065 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.422184 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.422301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.422418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.422535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.422652 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.422778 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.422896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.423014 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.423130 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.423247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.423364 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.423490 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.423610 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.423727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.423844 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.423961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.424078 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.424209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.424329 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.424446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.424563 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.424682 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.424798 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.424924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.425042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.425168 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.425285 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.425402 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.425518 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.425648 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.425768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.425885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.426002 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.426118 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.426235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.426361 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.426479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.426597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.426714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.426831 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.426955 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.427082 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] policy.driver = sql {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.427201 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] policy.driver = sql {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.427319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.427435 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.427552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.427670 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.427797 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.427917 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.428035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] receipt.caching = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.428153 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] receipt.caching = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.428270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.428388 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.428513 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.428632 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.428750 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.428869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.428987 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.429138 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.429271 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.429394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.429513 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] resource.caching = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.429630 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] resource.caching = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.429747 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.429864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.430060 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] resource.driver = sql {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.430182 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] resource.driver = sql {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.430301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.430418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.430536 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.430653 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.430779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.430947 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.431071 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] revoke.caching = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.431189 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] revoke.caching = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.431306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.431424 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.431550 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.431669 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.431786 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] role.cache_time = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.431904 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] role.cache_time = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.432023 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] role.caching = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.432140 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] role.caching = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.432266 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] role.driver = sql {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.432385 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] role.driver = sql {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.432503 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] role.list_limit = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.432621 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] role.list_limit = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.432739 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.432855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.432982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.433117 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.433238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.433355 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.433473 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.433591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.433722 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.433841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.433960 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.434077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.434196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.434322 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.434441 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.434558 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.434676 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.434793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.434927 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.435056 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.435167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.435294 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.435414 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.435583 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.435760 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.435897 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.436018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.436146 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.436281 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.436400 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.436519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.436646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.436767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.436885 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.437002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.437193 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.437316 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.437450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.437570 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.437688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.437807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.437924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.438041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.438168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.438289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.438406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.438524 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.438640 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.438757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.438883 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.439002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.439120 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.439237 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.439359 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.439485 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.439604 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.439721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.439838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.439956 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.440073 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.440199 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.440319 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.440437 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] token.cache_time = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.440553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] token.cache_time = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.440670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] token.caching = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.440786 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] token.caching = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.440911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.441030 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.441157 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] token.provider = fernet {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.441274 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] token.provider = fernet {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.441392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.441508 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.441638 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.441758 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.441877 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.441994 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.442112 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.442228 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.442354 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.442474 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.442592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.442709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.442826 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] trust.driver = sql {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.442943 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] trust.driver = sql {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.443069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.443188 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.443305 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.443421 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.443539 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.443655 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.443785 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.443904 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.444022 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.444140 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.444257 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.444257 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.444486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.444486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.444710 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.444710 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.connection = **** {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.444938 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.445133 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.445257 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.connection = **** {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.445390 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.445519 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.445647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.445766 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.445883 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.446000 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.446117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.446234 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.446359 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.446478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.446595 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.446713 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.446830 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.446948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.447073 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.447192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.447309 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.447428 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.447545 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.447661 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.447788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.447907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.448025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.448142 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.448259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.448376 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.448501 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.448620 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.448737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.448855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.448972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.449102 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.449305 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.449424 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.449525 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=103663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 20:24:41.449624 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:24:41.449624 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=103664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 20:24:41.479703 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.479928 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.480019 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.480160 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.480248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.480402 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.489093 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.489373 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.518029 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.518252 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.518672 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.518911 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.662193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.662398 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.711564 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.711798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.712413 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.712656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.777386 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.777581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.777698 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.777852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.777979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.778066 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.778145 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.778224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.778313 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.778441 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.778526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.779175 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.805263 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.805490 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.805671 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.805770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.805907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.805992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.806125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.806208 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.806340 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.806425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.806552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.806634 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.806767 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.806981 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.818060 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.818244 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.831021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.831254 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.831396 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.831503 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.831584 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.831732 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.831828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.831949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.832033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.832165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.832258 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.832377 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=103663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.832477 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.832610 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=103664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:24:41.850685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.850892 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 20:24:41.851234 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.851336 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.851417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 20:24:41.851494 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 20:24:41.851691 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.851834 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.851974 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.852059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 20:24:41.852213 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.852298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.852472 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.852554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.852781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.853044 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.864177 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.864455 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.864724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.864970 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.865058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.865291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.865561 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.865845 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.873216 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=103663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.874067 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=103664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.876490 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=103663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.876771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.877058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.877388 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=103664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.877488 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.877675 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.877761 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.877961 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.878045 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.878258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.878342 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.878536 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.878621 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.878813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.878895 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.879090 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.879174 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.879365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.879447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.879666 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.879945 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.880215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.906548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.907466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.908190 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=103663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.908473 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.909140 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=103664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.909428 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.913856 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=103663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.914086 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.914370 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.914741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=103664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.915025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.915311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.921746 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=103663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.922034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.922731 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=103664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.923012 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.926662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=103663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.926954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.927243 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.927526 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.927656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=103664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.927805 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.927947 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.928082 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.928235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.928352 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.928526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.928612 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.928806 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.929106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.929383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.929655 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.947971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.948258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.949138 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.949439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.953435 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=103663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.954870 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=103664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.957108 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.957407 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.958523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.958823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.962434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=103663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.963866 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=103664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.966115 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=103663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.967585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=103664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.969908 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.970211 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.970506 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.970793 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.971079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.971373 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.971463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.971694 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.971999 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.972287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.972572 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.972857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.990594 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=103663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.990978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.991252 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.991521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.992139 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=103664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:41.992528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.992803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:41.993087 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.000028 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.001668 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.001758 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.002029 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.003393 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.003669 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.005110 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=103663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:42.005399 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.005677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.005957 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.006256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.006794 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=103664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:42.007074 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.007353 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.007634 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.007982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.020352 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=103663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:42.020637 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.020932 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.021262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.021550 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.021836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.022119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=103664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:42.022216 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.022412 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.022714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.022996 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.023281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.023570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.023851 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.041854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=103663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:42.043490 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=103664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:42.045839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=103663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:42.047489 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=103664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:42.049297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.050943 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.051090 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.052609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=103663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:42.052728 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.052899 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.053211 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.054424 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=103664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:42.054706 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.054989 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.061336 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.063042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=103663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:42.063247 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.064934 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=103664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:42.066336 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.066633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.066921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.067206 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.068219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.068524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.068814 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.069115 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.076602 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=103663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:42.076896 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.077221 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.078536 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=103664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:42.078838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.079131 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.085216 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=103663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:42.085512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.085793 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.086067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.086339 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.086620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.086903 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.087173 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.087256 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=103664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:24:42.087444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.087527 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.087728 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.087820 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.088003 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.088094 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.088263 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.088356 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.088528 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.088612 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.088807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=103663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.088956 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.089115 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.089362 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.089635 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.089923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.090182 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.090460 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.090732 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=103664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:24:42.128379 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x563d841729c0 pid: 103663 (default app) Jul 21 20:24:42.130084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x563d841729c0 pid: 103664 (default app) Jul 21 20:24:42.131863 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75050e57-4194-4c54-a32e-4efb8352fcc8 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:42.132156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75050e57-4194-4c54-a32e-4efb8352fcc8 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:42.132448 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75050e57-4194-4c54-a32e-4efb8352fcc8 None None] PATH_INFO: `/v3` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:42.136689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1/1] 10.222.0.71 () {50 vars in 791 bytes} [Thu Jul 21 20:24:42 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:24:49.852531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2496cf-0230-42a8-9cd4-66baa59dea73 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:49.853088 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2496cf-0230-42a8-9cd4-66baa59dea73 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:49.853088 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2496cf-0230-42a8-9cd4-66baa59dea73 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:49.858059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1/2] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:24:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:24:49.863854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ac944-69c8-4434-9b26-7d3471efda45 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:49.864409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ac944-69c8-4434-9b26-7d3471efda45 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:49.864409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ac944-69c8-4434-9b26-7d3471efda45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:49.933849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: INFO dbcounter [None req-4e7ac944-69c8-4434-9b26-7d3471efda45 None None] Registered counter for database keystone Jul 21 20:24:49.955647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG oslo_db.sqlalchemy.engines [None req-4e7ac944-69c8-4434-9b26-7d3471efda45 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103663) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 20:24:49.962429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writer thread running {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 20:24:50.146642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG passlib.handlers.bcrypt [None req-4e7ac944-69c8-4434-9b26-7d3471efda45 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=103663) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 20:24:50.146972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG passlib.handlers.bcrypt [None req-4e7ac944-69c8-4434-9b26-7d3471efda45 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=103663) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 20:24:50.202955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-4e7ac944-69c8-4434-9b26-7d3471efda45 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:24:50.204612 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-4e7ac944-69c8-4434-9b26-7d3471efda45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:24:50.331242 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 2/3] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:24:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 21 20:24:50.421358 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2496cf-0230-42a8-9cd4-66baa59dea73 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:24:50.491944 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: INFO dbcounter [None req-fb2496cf-0230-42a8-9cd4-66baa59dea73 None None] Registered counter for database keystone Jul 21 20:24:50.511992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG oslo_db.sqlalchemy.engines [None req-fb2496cf-0230-42a8-9cd4-66baa59dea73 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103664) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 20:24:50.518702 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writer thread running {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 20:24:50.636982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2496cf-0230-42a8-9cd4-66baa59dea73 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:24:50.641300 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59759d5f-ad3a-40db-9b68-56c30fec86c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:24:50.641971 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59759d5f-ad3a-40db-9b68-56c30fec86c9 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:50.642254 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59759d5f-ad3a-40db-9b68-56c30fec86c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:50.642527 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59759d5f-ad3a-40db-9b68-56c30fec86c9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:50.651055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59759d5f-ad3a-40db-9b68-56c30fec86c9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:24:50.703731 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: WARNING keystone.common.rbac_enforcer.enforcer [None req-59759d5f-ad3a-40db-9b68-56c30fec86c9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 20:24:50.703998 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59759d5f-ad3a-40db-9b68-56c30fec86c9 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:24:50.714517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 2/4] 10.222.0.71 () {58 vars in 1324 bytes} [Thu Jul 21 20:24:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 20:24:50.721047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ac944-69c8-4434-9b26-7d3471efda45 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:24:50.737488 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ac944-69c8-4434-9b26-7d3471efda45 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:24:50.742026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a1410-b883-4f3d-b61b-9b4ea6f341f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:24:50.743178 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a1410-b883-4f3d-b61b-9b4ea6f341f0 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:50.743469 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a1410-b883-4f3d-b61b-9b4ea6f341f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:50.743749 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a1410-b883-4f3d-b61b-9b4ea6f341f0 admin admin] PATH_INFO: `/v3/projects/3dd46499ce284e948f969ddbeb31196c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:50.748508 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a1410-b883-4f3d-b61b-9b4ea6f341f0 admin admin] RBAC: Authorizing `identity:get_project(project_id=3dd46499ce284e948f969ddbeb31196c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:24:50.802741 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: WARNING keystone.common.rbac_enforcer.enforcer [None req-eb1a1410-b883-4f3d-b61b-9b4ea6f341f0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 20:24:50.803103 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a1410-b883-4f3d-b61b-9b4ea6f341f0 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:24:50.808114 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 3/5] 10.222.0.71 () {56 vars in 1207 bytes} [Thu Jul 21 20:24:50 2022] GET /identity/v3/projects/3dd46499ce284e948f969ddbeb31196c => generated 344 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:24:52.902990 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b9148-981d-4405-b5d6-4bab4657f5b4 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:52.903503 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b9148-981d-4405-b5d6-4bab4657f5b4 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:52.903503 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b9148-981d-4405-b5d6-4bab4657f5b4 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:52.908190 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 3/6] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:24:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:24:52.914528 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1e560-5783-4d4c-abc8-cac613a4e8cb None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:52.915044 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1e560-5783-4d4c-abc8-cac613a4e8cb None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:52.915133 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1e560-5783-4d4c-abc8-cac613a4e8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:52.934484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-d9e1e560-5783-4d4c-abc8-cac613a4e8cb None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:24:52.936616 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-d9e1e560-5783-4d4c-abc8-cac613a4e8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:24:52.945113 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 4/7] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:24:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 20:24:52.984058 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b9148-981d-4405-b5d6-4bab4657f5b4 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:24:53.004764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b9148-981d-4405-b5d6-4bab4657f5b4 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:24:53.011676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841afe2e-6be6-43a1-8acd-58fbf6e216c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:24:53.012812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841afe2e-6be6-43a1-8acd-58fbf6e216c5 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:53.013407 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841afe2e-6be6-43a1-8acd-58fbf6e216c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:53.013871 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841afe2e-6be6-43a1-8acd-58fbf6e216c5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:53.026906 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841afe2e-6be6-43a1-8acd-58fbf6e216c5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:24:53.030862 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841afe2e-6be6-43a1-8acd-58fbf6e216c5 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:24:53.045380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 4/8] 10.222.0.71 () {58 vars in 1324 bytes} [Thu Jul 21 20:24:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 20:24:53.051209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1e560-5783-4d4c-abc8-cac613a4e8cb None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:24:53.060820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1e560-5783-4d4c-abc8-cac613a4e8cb None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:24:53.065313 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49a4cb-b5e4-4dd8-930a-394ac9ae248a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:24:53.066191 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49a4cb-b5e4-4dd8-930a-394ac9ae248a admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:53.066478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49a4cb-b5e4-4dd8-930a-394ac9ae248a admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:53.066756 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49a4cb-b5e4-4dd8-930a-394ac9ae248a admin admin] PATH_INFO: `/v3/users/891d3014859343d8a558b44fef8a2f4f` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:53.068175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49a4cb-b5e4-4dd8-930a-394ac9ae248a admin admin] RBAC: Authorizing `identity:get_user(user_id=891d3014859343d8a558b44fef8a2f4f)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:24:53.070097 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49a4cb-b5e4-4dd8-930a-394ac9ae248a admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:24:53.071743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 5/9] 10.222.0.71 () {56 vars in 1198 bytes} [Thu Jul 21 20:24:53 2022] GET /identity/v3/users/891d3014859343d8a558b44fef8a2f4f => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:24:55.193429 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399afd07-a5c3-42e5-8cba-45a11b964bd1 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:55.193951 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399afd07-a5c3-42e5-8cba-45a11b964bd1 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:55.194054 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399afd07-a5c3-42e5-8cba-45a11b964bd1 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:55.195338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 5/10] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:24:55.201296 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffd90a-6b34-4f39-a120-9db5b400fbc5 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:55.201735 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffd90a-6b34-4f39-a120-9db5b400fbc5 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:55.201998 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffd90a-6b34-4f39-a120-9db5b400fbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:55.218275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-40ffd90a-6b34-4f39-a120-9db5b400fbc5 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:24:55.219441 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-40ffd90a-6b34-4f39-a120-9db5b400fbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:24:55.239892 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc9ea7-463a-4304-b3cc-84b22f589fc7 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:55.240180 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc9ea7-463a-4304-b3cc-84b22f589fc7 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:55.240462 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc9ea7-463a-4304-b3cc-84b22f589fc7 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:55.241936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 6/11] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:24:55.246866 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b093d0-30f0-4fc5-8998-dfd4b0452809 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:55.247154 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b093d0-30f0-4fc5-8998-dfd4b0452809 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:55.247930 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b093d0-30f0-4fc5-8998-dfd4b0452809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:55.287821 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG passlib.handlers.bcrypt [None req-60b093d0-30f0-4fc5-8998-dfd4b0452809 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=103664) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 20:24:55.288241 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG passlib.handlers.bcrypt [None req-60b093d0-30f0-4fc5-8998-dfd4b0452809 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=103664) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 20:24:55.300378 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 6/12] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:24:55.323508 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ffd90a-6b34-4f39-a120-9db5b400fbc5 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:24:55.334204 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-60b093d0-30f0-4fc5-8998-dfd4b0452809 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:24:55.335802 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-60b093d0-30f0-4fc5-8998-dfd4b0452809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:24:55.347560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:24:55.427516 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ffd90a-6b34-4f39-a120-9db5b400fbc5 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:24:55.434817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 7/13] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:24:55.458593 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b093d0-30f0-4fc5-8998-dfd4b0452809 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:24:55.465573 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747e1a5-f635-4c79-bc80-fd838e6b0c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:24:55.466388 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747e1a5-f635-4c79-bc80-fd838e6b0c78 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:55.466696 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747e1a5-f635-4c79-bc80-fd838e6b0c78 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:55.466955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747e1a5-f635-4c79-bc80-fd838e6b0c78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:55.479179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747e1a5-f635-4c79-bc80-fd838e6b0c78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:24:55.490944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-4747e1a5-f635-4c79-bc80-fd838e6b0c78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:24:55.490944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:24:55.490944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:24:55.490944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:24:55.490944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:24:55.490944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:24:55.490944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:24:55.490944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:24:55.490944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:24:55.490944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:24:55.490944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:24:55.490944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:24:55.490944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:24:55.490944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:24:55.490944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:24:55.490944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:24:55.491940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:24:55.491940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:24:55.491940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:24:55.491940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:24:55.491940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:24:55.491940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:24:55.491940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:24:55.491940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:24:55.491940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:24:55.491940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:24:55.491940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:24:55.491940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:24:55.491940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:24:55.491940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:24:55.491940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:24:55.491940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:24:55.492995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:24:55.492995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:24:55.492995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:24:55.492995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:24:55.492995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:24:55.492995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:24:55.492995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:24:55.492995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:24:55.492995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:24:55.492995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:24:55.492995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:24:55.492995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:24:55.492995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 7/14] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:24:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:24:55.498570 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747e1a5-f635-4c79-bc80-fd838e6b0c78 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:24:55.562737 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b093d0-30f0-4fc5-8998-dfd4b0452809 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:24:55.565046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:24:55.581262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:24:55.599846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747e1a5-f635-4c79-bc80-fd838e6b0c78 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:24:55.601532 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623ba45-37e6-4b16-a787-6b53b0c79b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:24:55.602442 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623ba45-37e6-4b16-a787-6b53b0c79b15 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:55.602730 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623ba45-37e6-4b16-a787-6b53b0c79b15 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:55.603008 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623ba45-37e6-4b16-a787-6b53b0c79b15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:55.634693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22f89c-e336-4902-bf82-47f7030449ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:24:55.635569 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22f89c-e336-4902-bf82-47f7030449ad None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:55.635851 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22f89c-e336-4902-bf82-47f7030449ad None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:55.636125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22f89c-e336-4902-bf82-47f7030449ad None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:55.636878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22f89c-e336-4902-bf82-47f7030449ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:24:55.637231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22f89c-e336-4902-bf82-47f7030449ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:24:55.638705 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22f89c-e336-4902-bf82-47f7030449ad None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:24:55.650267 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 8/15] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:24:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:24:55.655796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22f89c-e336-4902-bf82-47f7030449ad None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:24:55.670105 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d623ba45-37e6-4b16-a787-6b53b0c79b15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:24:55.671651 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d623ba45-37e6-4b16-a787-6b53b0c79b15 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:24:55.756094 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22f89c-e336-4902-bf82-47f7030449ad None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:24:55.769732 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 8/16] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:24:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 20:24:55.775627 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623ba45-37e6-4b16-a787-6b53b0c79b15 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:24:55.778495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:24:55.789136 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01ae52-2e30-4e4a-8e81-11489e999421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:24:55.790004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01ae52-2e30-4e4a-8e81-11489e999421 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:55.790287 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01ae52-2e30-4e4a-8e81-11489e999421 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:55.790562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01ae52-2e30-4e4a-8e81-11489e999421 None admin] PATH_INFO: `/v3/users/891d3014859343d8a558b44fef8a2f4f` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:55.791757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01ae52-2e30-4e4a-8e81-11489e999421 None admin] RBAC: Authorizing `identity:get_user(user_id=891d3014859343d8a558b44fef8a2f4f)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:24:55.793246 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01ae52-2e30-4e4a-8e81-11489e999421 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:24:55.795357 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 9/17] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:24:55 2022] GET /identity/v3/users/891d3014859343d8a558b44fef8a2f4f => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:24:55.800583 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01ae52-2e30-4e4a-8e81-11489e999421 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:24:55.814530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:24:55.874048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623ba45-37e6-4b16-a787-6b53b0c79b15 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:24:55.900328 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01ae52-2e30-4e4a-8e81-11489e999421 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:24:55.908030 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36f51e-b1e7-4861-86a2-0238eaf6df2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:24:55.908821 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36f51e-b1e7-4861-86a2-0238eaf6df2b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:55.909135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36f51e-b1e7-4861-86a2-0238eaf6df2b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:55.909420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36f51e-b1e7-4861-86a2-0238eaf6df2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:55.928939 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36f51e-b1e7-4861-86a2-0238eaf6df2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:24:55.930450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36f51e-b1e7-4861-86a2-0238eaf6df2b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:24:55.934008 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a271e71-ceb9-4f52-82d6-ac36f09ed49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:24:55.934806 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a271e71-ceb9-4f52-82d6-ac36f09ed49d None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:55.935616 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a271e71-ceb9-4f52-82d6-ac36f09ed49d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:55.935888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a271e71-ceb9-4f52-82d6-ac36f09ed49d None admin] PATH_INFO: `/v3/domains/default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:55.937151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a271e71-ceb9-4f52-82d6-ac36f09ed49d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:24:55.938570 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a271e71-ceb9-4f52-82d6-ac36f09ed49d None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:24:55.940187 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 10/18] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:24:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:24:55.946269 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a271e71-ceb9-4f52-82d6-ac36f09ed49d None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:24:55.952167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-5c36f51e-b1e7-4861-86a2-0238eaf6df2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:24:55.952167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:24:55.952167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:24:55.952167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:24:55.952167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:24:55.952167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:24:55.952167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:24:55.952167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:24:55.952167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:24:55.952167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:24:55.952167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:24:55.952167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:24:55.952167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:24:55.952167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:24:55.952167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:24:55.953022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:24:55.953022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:24:55.953022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:24:55.953022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:24:55.953022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:24:55.953022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:24:55.953022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:24:55.953022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:24:55.953022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:24:55.953022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:24:55.953022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:24:55.953022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:24:55.953022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:24:55.953022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:24:55.953022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:24:55.953022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:24:55.953022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:24:55.953871 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:24:55.953871 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:24:55.953871 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:24:55.953871 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:24:55.953871 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 9/19] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:24:55.959154 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36f51e-b1e7-4861-86a2-0238eaf6df2b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:24:56.017013 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:24:56.028339 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:24:56.048209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a271e71-ceb9-4f52-82d6-ac36f09ed49d None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:24:56.066911 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36f51e-b1e7-4861-86a2-0238eaf6df2b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:24:56.085337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb418b4-4b1f-4ae6-bada-8ca6d997952c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:24:56.085996 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb418b4-4b1f-4ae6-bada-8ca6d997952c None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:56.086279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb418b4-4b1f-4ae6-bada-8ca6d997952c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:56.086556 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb418b4-4b1f-4ae6-bada-8ca6d997952c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:56.087518 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb418b4-4b1f-4ae6-bada-8ca6d997952c None admin] RBAC: Adding query filter params (role.id=19f5e72c0be44a1197c7288de82e79e8, scope.domain.id=default, user.id=891d3014859343d8a558b44fef8a2f4f) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:24:56.087836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb418b4-4b1f-4ae6-bada-8ca6d997952c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:24:56.089325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb418b4-4b1f-4ae6-bada-8ca6d997952c None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:24:56.096483 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 11/20] 10.222.0.71 () {56 vars in 1322 bytes} [Thu Jul 21 20:24:55 2022] GET /identity/v3/role_assignments?user.id=891d3014859343d8a558b44fef8a2f4f&scope.domain.id=default&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:24:56.101840 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012313c5-d70e-472d-9150-58fb308543a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:24:56.102522 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012313c5-d70e-472d-9150-58fb308543a7 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:56.102806 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012313c5-d70e-472d-9150-58fb308543a7 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:56.103084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012313c5-d70e-472d-9150-58fb308543a7 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:56.103843 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012313c5-d70e-472d-9150-58fb308543a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:24:56.104147 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012313c5-d70e-472d-9150-58fb308543a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:24:56.105643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012313c5-d70e-472d-9150-58fb308543a7 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:24:56.126087 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 10/21] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:24:58.821404 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb1457-748b-4d55-8360-fe52fc7f9468 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:58.821883 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb1457-748b-4d55-8360-fe52fc7f9468 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:58.821883 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb1457-748b-4d55-8360-fe52fc7f9468 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:58.823422 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 12/22] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:24:58.842695 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b55ff-e4ed-44ea-8ace-feadcca7a258 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:58.843745 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b55ff-e4ed-44ea-8ace-feadcca7a258 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:58.843844 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b55ff-e4ed-44ea-8ace-feadcca7a258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:58.870125 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-393b55ff-e4ed-44ea-8ace-feadcca7a258 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:24:58.871357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-393b55ff-e4ed-44ea-8ace-feadcca7a258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:24:58.951882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ed631-fffe-4ae4-bdca-af6374be9dc8 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:58.952224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ed631-fffe-4ae4-bdca-af6374be9dc8 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:58.952447 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ed631-fffe-4ae4-bdca-af6374be9dc8 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:58.953857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 13/23] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:24:58.961333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb8364-1552-4e82-9e49-9f567fd683ab None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:58.961625 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb8364-1552-4e82-9e49-9f567fd683ab None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:58.961905 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb8364-1552-4e82-9e49-9f567fd683ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:58.983703 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 11/24] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:24:58.984872 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f292b-1a67-4f85-a2c6-4841cd003bc7 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:58.985191 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f292b-1a67-4f85-a2c6-4841cd003bc7 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:58.985481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f292b-1a67-4f85-a2c6-4841cd003bc7 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:58.986755 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 12/25] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:24:58.992049 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0357a-eddb-4d7d-9912-28ecd492d6b3 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:58.992343 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0357a-eddb-4d7d-9912-28ecd492d6b3 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:58.992631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0357a-eddb-4d7d-9912-28ecd492d6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:59.002175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-edbb8364-1552-4e82-9e49-9f567fd683ab None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:24:59.003399 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-edbb8364-1552-4e82-9e49-9f567fd683ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:24:59.024065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-dee0357a-eddb-4d7d-9912-28ecd492d6b3 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:24:59.025305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-dee0357a-eddb-4d7d-9912-28ecd492d6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:24:59.107647 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:24:59.130816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 14/26] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:24:59.131662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb8364-1552-4e82-9e49-9f567fd683ab None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:24:59.156467 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 13/27] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:24:59.166724 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0357a-eddb-4d7d-9912-28ecd492d6b3 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:24:59.246380 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:24:59.274978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb8364-1552-4e82-9e49-9f567fd683ab None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:24:59.298165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0357a-eddb-4d7d-9912-28ecd492d6b3 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:24:59.313576 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d06cf2-7f59-4702-8824-b3dc50ecfd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:24:59.314753 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d06cf2-7f59-4702-8824-b3dc50ecfd69 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:59.315271 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d06cf2-7f59-4702-8824-b3dc50ecfd69 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:59.315895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d06cf2-7f59-4702-8824-b3dc50ecfd69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:59.328848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d06cf2-7f59-4702-8824-b3dc50ecfd69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:24:59.330671 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d06cf2-7f59-4702-8824-b3dc50ecfd69 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:24:59.336211 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0fc67-62cc-4818-bef9-845fb21152e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:24:59.336894 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0fc67-62cc-4818-bef9-845fb21152e7 None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:59.337327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0fc67-62cc-4818-bef9-845fb21152e7 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:59.337608 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0fc67-62cc-4818-bef9-845fb21152e7 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:59.338415 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0fc67-62cc-4818-bef9-845fb21152e7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:24:59.339907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0fc67-62cc-4818-bef9-845fb21152e7 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:24:59.355919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-90d06cf2-7f59-4702-8824-b3dc50ecfd69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:24:59.355919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:24:59.355919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:24:59.355919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:24:59.355919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:24:59.355919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:24:59.355919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:24:59.355919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:24:59.355919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:24:59.355919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:24:59.355919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:24:59.355919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:24:59.355919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:24:59.355919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:24:59.355919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:24:59.356833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:24:59.356833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:24:59.356833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:24:59.356833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:24:59.356833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:24:59.356833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:24:59.356833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:24:59.356833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:24:59.356833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:24:59.356833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:24:59.356833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:24:59.356833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:24:59.356833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:24:59.356833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:24:59.356833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:24:59.356833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:24:59.356833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:24:59.357745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:24:59.357745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:24:59.357745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:24:59.357745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:24:59.357745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 15/28] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:24:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:24:59.358573 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708d50e-61a3-4d16-a716-dc9ff36c9772 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:59.358859 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708d50e-61a3-4d16-a716-dc9ff36c9772 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:59.359141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708d50e-61a3-4d16-a716-dc9ff36c9772 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:59.360421 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 16/29] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:24:59.366191 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708d50e-61a3-4d16-a716-dc9ff36c9772 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:24:59.484122 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708d50e-61a3-4d16-a716-dc9ff36c9772 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:24:59.493713 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 14/30] 10.222.0.71 () {60 vars in 1131 bytes} [Thu Jul 21 20:24:59 2022] POST /identity/v3/roles => generated 225 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:24:59.495106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0fc67-62cc-4818-bef9-845fb21152e7 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:24:59.515613 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:24:59.540984 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85272461-b3b6-42de-a02e-e4645fe413ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:24:59.542014 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85272461-b3b6-42de-a02e-e4645fe413ce None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:59.542633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85272461-b3b6-42de-a02e-e4645fe413ce None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:59.542759 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:24:59.543201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85272461-b3b6-42de-a02e-e4645fe413ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:59.571097 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85272461-b3b6-42de-a02e-e4645fe413ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:24:59.587769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-85272461-b3b6-42de-a02e-e4645fe413ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:24:59.587769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:24:59.587769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:24:59.587769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:24:59.587769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:24:59.587769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:24:59.587769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:24:59.587769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:24:59.587769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:24:59.587769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:24:59.587769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:24:59.587769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:24:59.587769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:24:59.587769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:24:59.587769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:24:59.587769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:24:59.588897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:24:59.588897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:24:59.588897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:24:59.588897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:24:59.588897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:24:59.588897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:24:59.588897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:24:59.588897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:24:59.588897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:24:59.588897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:24:59.588897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:24:59.588897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:24:59.588897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:24:59.588897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:24:59.588897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:24:59.588897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:24:59.589901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:24:59.589901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:24:59.589901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:24:59.589901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:24:59.589901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:24:59.589901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:24:59.589901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:24:59.589901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:24:59.589901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:24:59.589901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:24:59.589901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:24:59.589901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:24:59.596281 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 17/31] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:24:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:24:59.596281 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910b570-f16b-4784-ab06-ad9963944547 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:59.596281 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910b570-f16b-4784-ab06-ad9963944547 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:59.596281 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910b570-f16b-4784-ab06-ad9963944547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:59.645487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-b910b570-f16b-4784-ab06-ad9963944547 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:24:59.647221 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-b910b570-f16b-4784-ab06-ad9963944547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:24:59.666913 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0fc67-62cc-4818-bef9-845fb21152e7 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:24:59.726202 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeda83b-ece9-4c41-a655-abeb8a2eaba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:24:59.726884 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeda83b-ece9-4c41-a655-abeb8a2eaba4 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:59.727176 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeda83b-ece9-4c41-a655-abeb8a2eaba4 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:59.727459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeda83b-ece9-4c41-a655-abeb8a2eaba4 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:59.728204 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeda83b-ece9-4c41-a655-abeb8a2eaba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:24:59.728514 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeda83b-ece9-4c41-a655-abeb8a2eaba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:24:59.730942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeda83b-ece9-4c41-a655-abeb8a2eaba4 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:24:59.749001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 15/32] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:24:59.750771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeda83b-ece9-4c41-a655-abeb8a2eaba4 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:24:59.772927 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 18/33] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:24:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:24:59.772927 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910b570-f16b-4784-ab06-ad9963944547 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:24:59.879134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:24:59.886380 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910b570-f16b-4784-ab06-ad9963944547 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:24:59.888172 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeda83b-ece9-4c41-a655-abeb8a2eaba4 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:24:59.904886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:24:59.927630 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c1d53-b9b1-4506-9f1f-4d718f3aef34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:24:59.928310 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c1d53-b9b1-4506-9f1f-4d718f3aef34 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:59.928598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c1d53-b9b1-4506-9f1f-4d718f3aef34 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:59.928924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c1d53-b9b1-4506-9f1f-4d718f3aef34 None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:59.929644 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c1d53-b9b1-4506-9f1f-4d718f3aef34 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:24:59.930540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91957086-4a46-4d1d-9454-73b1d6f03298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:24:59.931130 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c1d53-b9b1-4506-9f1f-4d718f3aef34 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:24:59.931651 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91957086-4a46-4d1d-9454-73b1d6f03298 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:24:59.932135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91957086-4a46-4d1d-9454-73b1d6f03298 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:24:59.932624 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91957086-4a46-4d1d-9454-73b1d6f03298 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:24:59.933816 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91957086-4a46-4d1d-9454-73b1d6f03298 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:24:59.934335 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91957086-4a46-4d1d-9454-73b1d6f03298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:24:59.936412 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91957086-4a46-4d1d-9454-73b1d6f03298 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:24:59.951237 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 16/34] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:24:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:24:59.952444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91957086-4a46-4d1d-9454-73b1d6f03298 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:00.068116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91957086-4a46-4d1d-9454-73b1d6f03298 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:00.077930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 19/35] 10.222.0.71 () {60 vars in 1141 bytes} [Thu Jul 21 20:24:59 2022] POST /identity/v3/projects => generated 301 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:25:00.086502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c1d53-b9b1-4506-9f1f-4d718f3aef34 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:00.110583 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2e297-5451-4ed3-b5b5-666baef18c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:00.111675 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2e297-5451-4ed3-b5b5-666baef18c98 None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:00.112212 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2e297-5451-4ed3-b5b5-666baef18c98 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:00.112715 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2e297-5451-4ed3-b5b5-666baef18c98 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:00.113761 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e2e297-5451-4ed3-b5b5-666baef18c98 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:00.115465 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e2e297-5451-4ed3-b5b5-666baef18c98 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:00.131824 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 17/36] 10.222.0.71 () {60 vars in 1131 bytes} [Thu Jul 21 20:24:59 2022] POST /identity/v3/roles => generated 219 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:25:00.284647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c1d53-b9b1-4506-9f1f-4d718f3aef34 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:00.337724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fc4d0-13d6-4c3c-b293-972beffd3e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:00.338624 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fc4d0-13d6-4c3c-b293-972beffd3e24 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:00.338912 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fc4d0-13d6-4c3c-b293-972beffd3e24 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:00.339193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fc4d0-13d6-4c3c-b293-972beffd3e24 None admin] PATH_INFO: `/v3/users/891d3014859343d8a558b44fef8a2f4f` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:00.345566 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fc4d0-13d6-4c3c-b293-972beffd3e24 None admin] RBAC: Authorizing `identity:get_user(user_id=891d3014859343d8a558b44fef8a2f4f)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:00.345566 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fc4d0-13d6-4c3c-b293-972beffd3e24 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:00.346196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 20/37] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:00 2022] GET /identity/v3/users/891d3014859343d8a558b44fef8a2f4f => generated 247 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:00.351581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2e297-5451-4ed3-b5b5-666baef18c98 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:00.377371 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:00.416863 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65ddd0-400b-4d61-87e2-e0f5a764bf7d None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:00.417218 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65ddd0-400b-4d61-87e2-e0f5a764bf7d None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:00.417470 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65ddd0-400b-4d61-87e2-e0f5a764bf7d None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:00.419002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 21/38] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:00.423972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c33327-3479-4ef5-b7fc-be3cfd8dbabf None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:00.424261 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c33327-3479-4ef5-b7fc-be3cfd8dbabf None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:00.424541 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c33327-3479-4ef5-b7fc-be3cfd8dbabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:00.449100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-25c33327-3479-4ef5-b7fc-be3cfd8dbabf None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:00.453642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-25c33327-3479-4ef5-b7fc-be3cfd8dbabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:00.492582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:00.494090 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2e297-5451-4ed3-b5b5-666baef18c98 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:00.534592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d6292-5bdb-4dfe-baee-4d93b3ec9871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:00.535474 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d6292-5bdb-4dfe-baee-4d93b3ec9871 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:00.535689 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d6292-5bdb-4dfe-baee-4d93b3ec9871 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:00.535975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d6292-5bdb-4dfe-baee-4d93b3ec9871 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:00.537636 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d6292-5bdb-4dfe-baee-4d93b3ec9871 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:00.539131 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d6292-5bdb-4dfe-baee-4d93b3ec9871 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:00.540811 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 18/39] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:25:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:00.541980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 22/40] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:00.542349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b023b4-172c-44d8-b204-5bf255500db4 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:00.542620 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b023b4-172c-44d8-b204-5bf255500db4 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:00.542969 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b023b4-172c-44d8-b204-5bf255500db4 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:00.544408 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 19/41] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:00.550060 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c33327-3479-4ef5-b7fc-be3cfd8dbabf None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:00.550291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb5058-f158-41ce-9e7f-296ec2a1f48a None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:00.550586 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb5058-f158-41ce-9e7f-296ec2a1f48a None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:00.550870 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb5058-f158-41ce-9e7f-296ec2a1f48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:00.568290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-dbdb5058-f158-41ce-9e7f-296ec2a1f48a None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:00.569898 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-dbdb5058-f158-41ce-9e7f-296ec2a1f48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:00.636149 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 20/42] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:00.637060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb5058-f158-41ce-9e7f-296ec2a1f48a None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:00.656797 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c33327-3479-4ef5-b7fc-be3cfd8dbabf None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:00.658734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:00.746420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb5058-f158-41ce-9e7f-296ec2a1f48a None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:00.782594 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95761629-47aa-4f62-8cfa-e420b3a54043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:00.783274 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95761629-47aa-4f62-8cfa-e420b3a54043 None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:00.783561 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95761629-47aa-4f62-8cfa-e420b3a54043 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:00.783843 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95761629-47aa-4f62-8cfa-e420b3a54043 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:00.784637 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95761629-47aa-4f62-8cfa-e420b3a54043 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:00.786124 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95761629-47aa-4f62-8cfa-e420b3a54043 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:00.798619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930eefb4-f700-4398-9dab-d8d8443cb4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:00.799498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930eefb4-f700-4398-9dab-d8d8443cb4b6 None admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:00.799782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930eefb4-f700-4398-9dab-d8d8443cb4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:00.800065 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930eefb4-f700-4398-9dab-d8d8443cb4b6 None admin] PATH_INFO: `/v3/domains/default/users/891d3014859343d8a558b44fef8a2f4f/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:00.813107 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 21/43] 10.222.0.71 () {60 vars in 1131 bytes} [Thu Jul 21 20:25:00 2022] POST /identity/v3/roles => generated 223 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:25:00.813107 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95761629-47aa-4f62-8cfa-e420b3a54043 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:00.813419 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930eefb4-f700-4398-9dab-d8d8443cb4b6 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=891d3014859343d8a558b44fef8a2f4f, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:00.813419 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930eefb4-f700-4398-9dab-d8d8443cb4b6 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:00.820610 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 23/44] 10.222.0.71 () {58 vars in 1361 bytes} [Thu Jul 21 20:25:00 2022] PUT /identity/v3/domains/default/users/891d3014859343d8a558b44fef8a2f4f/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:25:00.905966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:00.938787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95761629-47aa-4f62-8cfa-e420b3a54043 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:00.980568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac69424-67ae-4035-abf4-f6231b7e8f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:00.983903 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac69424-67ae-4035-abf4-f6231b7e8f49 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:00.983903 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac69424-67ae-4035-abf4-f6231b7e8f49 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:00.983903 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac69424-67ae-4035-abf4-f6231b7e8f49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:00.983903 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac69424-67ae-4035-abf4-f6231b7e8f49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:00.985287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac69424-67ae-4035-abf4-f6231b7e8f49 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:00.987289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 22/45] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:25:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:00.993866 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930eefb4-f700-4398-9dab-d8d8443cb4b6 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:01.024273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:01.099836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930eefb4-f700-4398-9dab-d8d8443cb4b6 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:01.133835 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb267f9-6166-45f7-806c-0651b9ff4581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:01.134529 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb267f9-6166-45f7-806c-0651b9ff4581 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:01.134816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb267f9-6166-45f7-806c-0651b9ff4581 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:01.135089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb267f9-6166-45f7-806c-0651b9ff4581 None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:01.135819 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb267f9-6166-45f7-806c-0651b9ff4581 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:01.137317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb267f9-6166-45f7-806c-0651b9ff4581 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:01.162862 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 24/46] 10.222.0.71 () {60 vars in 1141 bytes} [Thu Jul 21 20:25:00 2022] POST /identity/v3/projects => generated 312 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:25:02.903772 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d55e6-587d-42e2-b8de-0c995d671976 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:02.904259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d55e6-587d-42e2-b8de-0c995d671976 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:02.904259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d55e6-587d-42e2-b8de-0c995d671976 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:02.905663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 23/47] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:02.910785 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dece028-d4a1-4850-84c2-e7965b85d0d1 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:02.911213 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dece028-d4a1-4850-84c2-e7965b85d0d1 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:02.911351 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dece028-d4a1-4850-84c2-e7965b85d0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:02.927651 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-4dece028-d4a1-4850-84c2-e7965b85d0d1 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:02.928834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-4dece028-d4a1-4850-84c2-e7965b85d0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:03.002229 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 25/48] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:03.026266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d55e6-587d-42e2-b8de-0c995d671976 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:03.160819 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d55e6-587d-42e2-b8de-0c995d671976 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:03.201755 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047a18d-ac33-45ee-9105-d47992ed5f5c None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:03.202199 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047a18d-ac33-45ee-9105-d47992ed5f5c None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:03.202520 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047a18d-ac33-45ee-9105-d47992ed5f5c None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:03.204361 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 26/49] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:03.212044 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724caf7-decf-42dd-a583-25db1cff4b97 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:03.212494 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724caf7-decf-42dd-a583-25db1cff4b97 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:03.212935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724caf7-decf-42dd-a583-25db1cff4b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:03.214435 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6ae16-c8b6-4060-8f95-f6e542cf688c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:03.215238 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6ae16-c8b6-4060-8f95-f6e542cf688c None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:03.215531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6ae16-c8b6-4060-8f95-f6e542cf688c None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:03.215811 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6ae16-c8b6-4060-8f95-f6e542cf688c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:03.229298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af6ae16-c8b6-4060-8f95-f6e542cf688c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:03.234805 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-9724caf7-decf-42dd-a583-25db1cff4b97 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:03.236059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-9724caf7-decf-42dd-a583-25db1cff4b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:03.238170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:03.238785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-1af6ae16-c8b6-4060-8f95-f6e542cf688c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:03.238785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:03.238785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:03.238785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:03.238785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:03.238785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:03.238785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:03.238785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:03.238785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:03.238785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:03.238785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:03.238785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:03.238785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:03.238785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:03.238785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:03.238785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:03.239804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:03.239804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:03.239804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:03.239804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:03.239804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:03.239804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:03.239804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:03.239804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:03.239804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:03.239804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:03.239804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:03.239804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:03.239804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:03.239804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:03.239804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:03.239804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:03.241064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:03.241064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:03.241064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:03.241064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:03.241064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:03.241064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:03.241064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:03.241064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:03.241064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:03.241064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:03.241064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:03.241064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:03.241064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 24/50] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:25:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:03.245905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6ae16-c8b6-4060-8f95-f6e542cf688c None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:03.250884 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:03.317125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 27/51] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:03.340308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724caf7-decf-42dd-a583-25db1cff4b97 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:03.354621 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6ae16-c8b6-4060-8f95-f6e542cf688c None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:03.392542 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd1562-ee17-4883-ae7e-4f7bf9fc6203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:03.393247 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd1562-ee17-4883-ae7e-4f7bf9fc6203 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:03.393535 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd1562-ee17-4883-ae7e-4f7bf9fc6203 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:03.393820 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd1562-ee17-4883-ae7e-4f7bf9fc6203 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:03.394624 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd1562-ee17-4883-ae7e-4f7bf9fc6203 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:03.394894 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd1562-ee17-4883-ae7e-4f7bf9fc6203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:03.396301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd1562-ee17-4883-ae7e-4f7bf9fc6203 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:03.406943 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 25/52] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:03.412628 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd1562-ee17-4883-ae7e-4f7bf9fc6203 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:03.451853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724caf7-decf-42dd-a583-25db1cff4b97 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:03.485979 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:03.493425 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256ca0c-6199-4d39-bb77-d67697ae292d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:03.494231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256ca0c-6199-4d39-bb77-d67697ae292d None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:03.494510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256ca0c-6199-4d39-bb77-d67697ae292d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:03.494787 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256ca0c-6199-4d39-bb77-d67697ae292d None admin] PATH_INFO: `/v3/domains/default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:03.496189 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256ca0c-6199-4d39-bb77-d67697ae292d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:03.497769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256ca0c-6199-4d39-bb77-d67697ae292d None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:03.504218 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 28/53] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:25:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:03.505552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256ca0c-6199-4d39-bb77-d67697ae292d None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:03.515820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:03.627626 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256ca0c-6199-4d39-bb77-d67697ae292d None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:03.631037 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd1562-ee17-4883-ae7e-4f7bf9fc6203 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:03.663529 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf62f0-d2da-4b9a-b2e9-b2b9ee2c50cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:03.664110 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf62f0-d2da-4b9a-b2e9-b2b9ee2c50cc None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:03.664401 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf62f0-d2da-4b9a-b2e9-b2b9ee2c50cc None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:03.664741 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf62f0-d2da-4b9a-b2e9-b2b9ee2c50cc None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:03.665437 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf62f0-d2da-4b9a-b2e9-b2b9ee2c50cc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:03.666249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78046be2-13bb-4df2-bbbc-c30169218a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:03.666947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf62f0-d2da-4b9a-b2e9-b2b9ee2c50cc None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:03.667134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78046be2-13bb-4df2-bbbc-c30169218a84 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:03.667424 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78046be2-13bb-4df2-bbbc-c30169218a84 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:03.667726 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78046be2-13bb-4df2-bbbc-c30169218a84 None admin] PATH_INFO: `/v3/users/891d3014859343d8a558b44fef8a2f4f` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:03.669288 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78046be2-13bb-4df2-bbbc-c30169218a84 None admin] RBAC: Authorizing `identity:get_user(user_id=891d3014859343d8a558b44fef8a2f4f)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:03.671250 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78046be2-13bb-4df2-bbbc-c30169218a84 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:03.672930 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 26/54] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:03 2022] GET /identity/v3/users/891d3014859343d8a558b44fef8a2f4f => generated 247 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:03.678514 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78046be2-13bb-4df2-bbbc-c30169218a84 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:03.694697 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 29/55] 10.222.0.71 () {60 vars in 1140 bytes} [Thu Jul 21 20:25:03 2022] POST /identity/v3/projects => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:25:03.803348 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78046be2-13bb-4df2-bbbc-c30169218a84 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:03.840214 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fc8fc-4b80-4db5-874b-44ed47ecdaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:03.841040 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fc8fc-4b80-4db5-874b-44ed47ecdaf9 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:03.841926 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fc8fc-4b80-4db5-874b-44ed47ecdaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:03.842208 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fc8fc-4b80-4db5-874b-44ed47ecdaf9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:03.843468 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fc8fc-4b80-4db5-874b-44ed47ecdaf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:03.844941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fc8fc-4b80-4db5-874b-44ed47ecdaf9 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:03.846663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 27/56] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:25:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:03.853952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf62f0-d2da-4b9a-b2e9-b2b9ee2c50cc None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:03.915023 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:03.960794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf62f0-d2da-4b9a-b2e9-b2b9ee2c50cc None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:03.995310 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a0f59-1f84-45d6-aa45-219c23aeb74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:03.995968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a0f59-1f84-45d6-aa45-219c23aeb74d None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:03.996263 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a0f59-1f84-45d6-aa45-219c23aeb74d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:03.996541 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a0f59-1f84-45d6-aa45-219c23aeb74d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:03.997442 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a0f59-1f84-45d6-aa45-219c23aeb74d None admin] RBAC: Adding query filter params (role.id=19f5e72c0be44a1197c7288de82e79e8, scope.domain.id=default, user.id=891d3014859343d8a558b44fef8a2f4f) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:03.997759 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a0f59-1f84-45d6-aa45-219c23aeb74d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:03.999218 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a0f59-1f84-45d6-aa45-219c23aeb74d None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:04.005082 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 30/57] 10.222.0.71 () {56 vars in 1322 bytes} [Thu Jul 21 20:25:03 2022] GET /identity/v3/role_assignments?user.id=891d3014859343d8a558b44fef8a2f4f&scope.domain.id=default&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:05.726676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1858d-e41e-4997-8fac-3579ab72855f None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:05.727182 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1858d-e41e-4997-8fac-3579ab72855f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:05.727182 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1858d-e41e-4997-8fac-3579ab72855f None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:05.728537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 28/58] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:05.733753 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e4d7c-272f-43fa-a3fe-69888aeb023a None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:05.734132 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e4d7c-272f-43fa-a3fe-69888aeb023a None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:05.734311 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e4d7c-272f-43fa-a3fe-69888aeb023a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:05.751100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-e48e4d7c-272f-43fa-a3fe-69888aeb023a None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:05.752270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-e48e4d7c-272f-43fa-a3fe-69888aeb023a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:05.816403 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 31/59] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:05.840699 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1858d-e41e-4997-8fac-3579ab72855f None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:05.843603 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:05.939388 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1858d-e41e-4997-8fac-3579ab72855f None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:05.973204 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da7743-f2a5-4440-8df5-9e7ed459ea44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:05.974261 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da7743-f2a5-4440-8df5-9e7ed459ea44 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:05.974541 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da7743-f2a5-4440-8df5-9e7ed459ea44 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:05.974818 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da7743-f2a5-4440-8df5-9e7ed459ea44 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:05.975985 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da7743-f2a5-4440-8df5-9e7ed459ea44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:05.977494 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da7743-f2a5-4440-8df5-9e7ed459ea44 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:05.979045 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 29/60] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:25:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:05.984716 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e4d7c-272f-43fa-a3fe-69888aeb023a None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:06.037560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:06.084124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e4d7c-272f-43fa-a3fe-69888aeb023a None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:06.116979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946b986-205c-4d7a-bafa-b3a727cd0450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:06.117670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946b986-205c-4d7a-bafa-b3a727cd0450 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:06.117946 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946b986-205c-4d7a-bafa-b3a727cd0450 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:06.118216 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946b986-205c-4d7a-bafa-b3a727cd0450 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:06.118914 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946b986-205c-4d7a-bafa-b3a727cd0450 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:06.120349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946b986-205c-4d7a-bafa-b3a727cd0450 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:06.145087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 32/61] 10.222.0.71 () {60 vars in 1132 bytes} [Thu Jul 21 20:25:05 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:25:09.266644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770c876-4f79-42d3-837e-a8c48874976f None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:09.267658 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770c876-4f79-42d3-837e-a8c48874976f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:09.268164 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770c876-4f79-42d3-837e-a8c48874976f None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:09.269761 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 30/62] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:09.275375 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca36d0-f30b-4d55-a637-a3fce3518f67 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:09.276234 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca36d0-f30b-4d55-a637-a3fce3518f67 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:09.276711 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca36d0-f30b-4d55-a637-a3fce3518f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:09.281137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3887e-e89b-4ba9-8380-95feb8bd8a93 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:09.281675 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3887e-e89b-4ba9-8380-95feb8bd8a93 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:09.282162 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3887e-e89b-4ba9-8380-95feb8bd8a93 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:09.283727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 31/63] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:09.288862 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc54ad0-4a70-4857-90e6-8a19124ee206 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:09.289439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc54ad0-4a70-4857-90e6-8a19124ee206 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:09.289925 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc54ad0-4a70-4857-90e6-8a19124ee206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:09.300855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-87ca36d0-f30b-4d55-a637-a3fce3518f67 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:09.303180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-87ca36d0-f30b-4d55-a637-a3fce3518f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:09.320424 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-fdc54ad0-4a70-4857-90e6-8a19124ee206 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:09.322028 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-fdc54ad0-4a70-4857-90e6-8a19124ee206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:09.440274 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 33/64] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:09.462755 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:09.464287 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca36d0-f30b-4d55-a637-a3fce3518f67 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:09.477670 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 32/65] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:09.522030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:09.530223 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc54ad0-4a70-4857-90e6-8a19124ee206 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:09.621679 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca36d0-f30b-4d55-a637-a3fce3518f67 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:09.660815 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef6b6b-03cc-4ff6-a5e9-64356e558fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:09.661916 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef6b6b-03cc-4ff6-a5e9-64356e558fdd None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:09.662346 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef6b6b-03cc-4ff6-a5e9-64356e558fdd None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:09.662787 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef6b6b-03cc-4ff6-a5e9-64356e558fdd None admin] PATH_INFO: `/v3/roles/member` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:09.675882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef6b6b-03cc-4ff6-a5e9-64356e558fdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:09.683958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-96ef6b6b-03cc-4ff6-a5e9-64356e558fdd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:09.683958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:09.683958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:09.683958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:09.683958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:09.683958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:09.683958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:09.683958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:09.683958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:09.683958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:09.683958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:09.683958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:09.683958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:09.683958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:09.683958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:09.683958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:09.685126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:09.685126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:09.685126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:09.685126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:09.685126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:09.685126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:09.685126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:09.685126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:09.685126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:09.685126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:09.685126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:09.685126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:09.685126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:09.685126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:09.685126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:09.685126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:09.686042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:09.686042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:09.686042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:09.686042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:09.686042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:09.686042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:09.686042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:09.686042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:09.686042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:09.686042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:09.686042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:09.686042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:09.687631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc54ad0-4a70-4857-90e6-8a19124ee206 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:09.688172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 34/66] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:25:09.689674 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0854b-c0c4-4d11-bf00-4afb749c567e None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:09.690115 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0854b-c0c4-4d11-bf00-4afb749c567e None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:09.690536 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0854b-c0c4-4d11-bf00-4afb749c567e None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:09.692238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 35/67] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:09.698011 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0854b-c0c4-4d11-bf00-4afb749c567e None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:09.741180 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe86652-7a23-4208-a151-0e9c38732b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:09.742157 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe86652-7a23-4208-a151-0e9c38732b0b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:09.742445 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe86652-7a23-4208-a151-0e9c38732b0b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:09.742751 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe86652-7a23-4208-a151-0e9c38732b0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:09.768958 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe86652-7a23-4208-a151-0e9c38732b0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:09.782764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-9fe86652-7a23-4208-a151-0e9c38732b0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:09.782764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:09.782764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:09.782764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:09.782764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:09.782764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:09.782764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:09.782764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:09.782764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:09.782764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:09.782764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:09.782764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:09.782764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:09.782764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:09.782764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:09.782764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:09.783893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:09.783893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:09.783893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:09.783893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:09.783893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:09.783893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:09.783893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:09.783893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:09.783893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:09.783893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:09.783893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:09.783893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:09.783893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:09.783893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:09.783893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:09.783893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:09.784839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:09.784839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:09.784839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:09.784839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:09.784839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:09.784839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:09.784839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:09.784839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:09.784839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:09.784839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:09.784839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:09.784839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:09.784839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 33/68] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:25:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:25:09.785525 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa86a8e-8a03-4448-b317-9cc72ca66485 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:09.785690 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa86a8e-8a03-4448-b317-9cc72ca66485 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:09.785972 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa86a8e-8a03-4448-b317-9cc72ca66485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:09.807504 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-4aa86a8e-8a03-4448-b317-9cc72ca66485 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:09.808997 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-4aa86a8e-8a03-4448-b317-9cc72ca66485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:09.835457 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:09.845672 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0854b-c0c4-4d11-bf00-4afb749c567e None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:09.887308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0aaa9-73ba-4dc0-bfda-e23398ab24d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:09.887803 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0aaa9-73ba-4dc0-bfda-e23398ab24d5 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:09.888633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0aaa9-73ba-4dc0-bfda-e23398ab24d5 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:09.888910 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0aaa9-73ba-4dc0-bfda-e23398ab24d5 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:09.889708 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0aaa9-73ba-4dc0-bfda-e23398ab24d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:09.890054 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0aaa9-73ba-4dc0-bfda-e23398ab24d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:09.891582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0aaa9-73ba-4dc0-bfda-e23398ab24d5 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:09.892335 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:09.907998 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 34/70] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:09.913187 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 36/70] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:09.913187 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0aaa9-73ba-4dc0-bfda-e23398ab24d5 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:09.915332 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa86a8e-8a03-4448-b317-9cc72ca66485 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:10.034843 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa86a8e-8a03-4448-b317-9cc72ca66485 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:10.037428 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0aaa9-73ba-4dc0-bfda-e23398ab24d5 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:10.073089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729b69b-7001-4bd3-a7b7-4f3f5ec9e171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:10.073302 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6accce-03c4-406e-8163-17794a932d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:10.073406 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729b69b-7001-4bd3-a7b7-4f3f5ec9e171 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:10.073485 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6accce-03c4-406e-8163-17794a932d75 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:10.073589 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729b69b-7001-4bd3-a7b7-4f3f5ec9e171 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:10.073670 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6accce-03c4-406e-8163-17794a932d75 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:10.073787 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729b69b-7001-4bd3-a7b7-4f3f5ec9e171 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:10.073872 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6accce-03c4-406e-8163-17794a932d75 None admin] PATH_INFO: `/v3/users/89583f5f42ab49058e780d3e5c44662c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:10.074006 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729b69b-7001-4bd3-a7b7-4f3f5ec9e171 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:10.074313 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729b69b-7001-4bd3-a7b7-4f3f5ec9e171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:10.076221 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729b69b-7001-4bd3-a7b7-4f3f5ec9e171 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:10.084835 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 37/71] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:10.085701 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729b69b-7001-4bd3-a7b7-4f3f5ec9e171 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:10.091923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6accce-03c4-406e-8163-17794a932d75 None admin] RBAC: Authorizing `identity:get_user(user_id=89583f5f42ab49058e780d3e5c44662c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:10.093324 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6accce-03c4-406e-8163-17794a932d75 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:10.095093 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 35/72] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:09 2022] GET /identity/v3/users/89583f5f42ab49058e780d3e5c44662c => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:10.096342 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052d250-bfb3-4dee-9b67-9e710279b310 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:10.096630 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052d250-bfb3-4dee-9b67-9e710279b310 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:10.096911 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052d250-bfb3-4dee-9b67-9e710279b310 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:10.098194 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 36/73] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:10.099522 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b61598-95ec-4357-898d-0ffc7c33265e None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:10.099894 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b61598-95ec-4357-898d-0ffc7c33265e None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:10.100079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b61598-95ec-4357-898d-0ffc7c33265e None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:10.101351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 37/74] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:10.102073 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b61598-95ec-4357-898d-0ffc7c33265e None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:10.115761 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:10.155145 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:10.196267 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729b69b-7001-4bd3-a7b7-4f3f5ec9e171 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:10.209502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b61598-95ec-4357-898d-0ffc7c33265e None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:10.231935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f62ff1f-7684-4476-8f76-fdba9b191581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:10.232747 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62ff1f-7684-4476-8f76-fdba9b191581 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:10.233027 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62ff1f-7684-4476-8f76-fdba9b191581 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:10.233339 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62ff1f-7684-4476-8f76-fdba9b191581 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:10.244214 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f62ff1f-7684-4476-8f76-fdba9b191581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:10.246008 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b5540-d884-494b-83de-2649a3fae2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:10.246944 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b5540-d884-494b-83de-2649a3fae2dc None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:10.247173 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b5540-d884-494b-83de-2649a3fae2dc None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:10.247980 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b5540-d884-494b-83de-2649a3fae2dc None admin] PATH_INFO: `/v3/users/891d3014859343d8a558b44fef8a2f4f` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:10.249460 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b5540-d884-494b-83de-2649a3fae2dc None admin] RBAC: Authorizing `identity:get_user(user_id=891d3014859343d8a558b44fef8a2f4f)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:10.251131 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b5540-d884-494b-83de-2649a3fae2dc None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:10.252588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 38/75] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:10 2022] GET /identity/v3/users/891d3014859343d8a558b44fef8a2f4f => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:10.252948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-7f62ff1f-7684-4476-8f76-fdba9b191581 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:10.252948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:10.252948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:10.252948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:10.252948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:10.252948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:10.252948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:10.252948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:10.252948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:10.252948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:10.252948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:10.252948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:10.252948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:10.252948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:10.252948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:10.252948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:10.254067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:10.254067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:10.254067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:10.254067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:10.254067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:10.254067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:10.254067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:10.254067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:10.254067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:10.254067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:10.254067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:10.254067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:10.254067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:10.254067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:10.254067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:10.254067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:10.254992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:10.254992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:10.254992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:10.254992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:10.254992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:10.254992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:10.254992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:10.254992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:10.254992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:10.254992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:10.254992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:10.254992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:10.255567 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b5540-d884-494b-83de-2649a3fae2dc None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:10.255641 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 38/76] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:25:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:10.255641 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3859f3a-0842-4f15-aeae-983cf394ed71 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:10.255641 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3859f3a-0842-4f15-aeae-983cf394ed71 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:10.256209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3859f3a-0842-4f15-aeae-983cf394ed71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:10.273188 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-a3859f3a-0842-4f15-aeae-983cf394ed71 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:10.274394 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a3859f3a-0842-4f15-aeae-983cf394ed71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:10.346560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 39/77] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:10.347753 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a963cad-d02a-47af-a3e9-6331f755580b None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:10.348040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a963cad-d02a-47af-a3e9-6331f755580b None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:10.348314 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a963cad-d02a-47af-a3e9-6331f755580b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:10.365343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-9a963cad-d02a-47af-a3e9-6331f755580b None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:10.366545 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-9a963cad-d02a-47af-a3e9-6331f755580b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:10.366653 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b5540-d884-494b-83de-2649a3fae2dc None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:10.375278 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:10.385139 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:10.404670 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfcefc7-40d4-48b7-8218-b1655d260c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:10.405378 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfcefc7-40d4-48b7-8218-b1655d260c96 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:10.405665 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfcefc7-40d4-48b7-8218-b1655d260c96 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:10.405959 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfcefc7-40d4-48b7-8218-b1655d260c96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:10.437616 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 40/78] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:10.438491 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a963cad-d02a-47af-a3e9-6331f755580b None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:10.482647 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfcefc7-40d4-48b7-8218-b1655d260c96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:10.484134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfcefc7-40d4-48b7-8218-b1655d260c96 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:10.546606 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a963cad-d02a-47af-a3e9-6331f755580b None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:10.582740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce072b-30ed-4ada-82a1-e942c6f7ef1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:10.583395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce072b-30ed-4ada-82a1-e942c6f7ef1d None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:10.583678 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce072b-30ed-4ada-82a1-e942c6f7ef1d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:10.583957 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce072b-30ed-4ada-82a1-e942c6f7ef1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:10.585328 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 39/79] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:10.586184 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfcefc7-40d4-48b7-8218-b1655d260c96 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:10.600603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:10.607688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:10.668690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce072b-30ed-4ada-82a1-e942c6f7ef1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:10.670756 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce072b-30ed-4ada-82a1-e942c6f7ef1d None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:10.692528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfcefc7-40d4-48b7-8218-b1655d260c96 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:10.726923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a34df-2dd3-4f7c-958b-1013f91ee9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:10.727598 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a34df-2dd3-4f7c-958b-1013f91ee9f1 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:10.727890 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a34df-2dd3-4f7c-958b-1013f91ee9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:10.728172 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a34df-2dd3-4f7c-958b-1013f91ee9f1 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:10.728941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a34df-2dd3-4f7c-958b-1013f91ee9f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:10.729284 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a34df-2dd3-4f7c-958b-1013f91ee9f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:10.730711 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a34df-2dd3-4f7c-958b-1013f91ee9f1 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:10.739419 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 40/80] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:25:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:10.740018 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a34df-2dd3-4f7c-958b-1013f91ee9f1 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:10.774329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 41/81] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:10.775179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce072b-30ed-4ada-82a1-e942c6f7ef1d None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:10.818233 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:10.818641 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:10.840596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a34df-2dd3-4f7c-958b-1013f91ee9f1 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:10.873528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1757c-6f71-407e-aef5-3f6cb2faa7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:10.874252 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1757c-6f71-407e-aef5-3f6cb2faa7b7 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:10.874543 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1757c-6f71-407e-aef5-3f6cb2faa7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:10.874817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1757c-6f71-407e-aef5-3f6cb2faa7b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:10.876015 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1757c-6f71-407e-aef5-3f6cb2faa7b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:10.876770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce072b-30ed-4ada-82a1-e942c6f7ef1d None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:10.877506 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1757c-6f71-407e-aef5-3f6cb2faa7b7 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:10.879141 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 41/82] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:25:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:10.879913 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1757c-6f71-407e-aef5-3f6cb2faa7b7 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:10.912223 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951e7b6-93cb-4ea3-876f-58194c405cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:10.913020 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951e7b6-93cb-4ea3-876f-58194c405cf4 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:10.913335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951e7b6-93cb-4ea3-876f-58194c405cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:10.913611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951e7b6-93cb-4ea3-876f-58194c405cf4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:10.923539 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951e7b6-93cb-4ea3-876f-58194c405cf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:10.931207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-4951e7b6-93cb-4ea3-876f-58194c405cf4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:10.931207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:10.931207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:10.931207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:10.931207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:10.931207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:10.931207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:10.931207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:10.931207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:10.931207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:10.931207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:10.931207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:10.931207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:10.931207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:10.931207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:10.931207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:10.932104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:10.932104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:10.932104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:10.932104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:10.932104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:10.932104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:10.932104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:10.932104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:10.932104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:10.932104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:10.932104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:10.932104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:10.932104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:10.932104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:10.932104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:10.932104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:10.932885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:10.932885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:10.932885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:10.932885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:10.932885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:10.932885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:10.932885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:10.932885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:10.932885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:10.932885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:10.932885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:10.932885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:10.932885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 42/83] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:10.933572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951e7b6-93cb-4ea3-876f-58194c405cf4 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:10.980989 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1757c-6f71-407e-aef5-3f6cb2faa7b7 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:11.015193 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061cdae-52a4-4687-8669-92d62f0a6628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:11.016055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061cdae-52a4-4687-8669-92d62f0a6628 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:11.016336 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061cdae-52a4-4687-8669-92d62f0a6628 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:11.016622 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061cdae-52a4-4687-8669-92d62f0a6628 None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:11.017847 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061cdae-52a4-4687-8669-92d62f0a6628 None admin] RBAC: Authorizing `identity:get_project(project_id=3790dd61247c450ebbf3ef2e90a818e0)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:11.019286 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061cdae-52a4-4687-8669-92d62f0a6628 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:11.020982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 42/84] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:10 2022] GET /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:11.025954 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061cdae-52a4-4687-8669-92d62f0a6628 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:11.037749 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:11.043396 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951e7b6-93cb-4ea3-876f-58194c405cf4 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:11.045056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:11.078919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d9aff-cac3-4fa5-96e7-c05fa5a6f066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:11.079797 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d9aff-cac3-4fa5-96e7-c05fa5a6f066 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:11.080090 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d9aff-cac3-4fa5-96e7-c05fa5a6f066 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:11.080363 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d9aff-cac3-4fa5-96e7-c05fa5a6f066 None admin] PATH_INFO: `/v3/users/89583f5f42ab49058e780d3e5c44662c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:11.081631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d9aff-cac3-4fa5-96e7-c05fa5a6f066 None admin] RBAC: Authorizing `identity:get_user(user_id=89583f5f42ab49058e780d3e5c44662c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:11.083095 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d9aff-cac3-4fa5-96e7-c05fa5a6f066 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:11.084750 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 43/85] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:10 2022] GET /identity/v3/users/89583f5f42ab49058e780d3e5c44662c => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:11.085535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d9aff-cac3-4fa5-96e7-c05fa5a6f066 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:11.138108 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061cdae-52a4-4687-8669-92d62f0a6628 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:11.174219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece91a22-41bb-4984-b7b2-b9d5cf636cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:11.175188 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece91a22-41bb-4984-b7b2-b9d5cf636cf7 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:11.175593 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece91a22-41bb-4984-b7b2-b9d5cf636cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:11.176060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece91a22-41bb-4984-b7b2-b9d5cf636cf7 None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:11.177613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece91a22-41bb-4984-b7b2-b9d5cf636cf7 None admin] RBAC: Authorizing `identity:get_project(project_id=3790dd61247c450ebbf3ef2e90a818e0)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:11.179422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece91a22-41bb-4984-b7b2-b9d5cf636cf7 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:11.181279 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 43/86] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:11 2022] GET /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:11.182065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece91a22-41bb-4984-b7b2-b9d5cf636cf7 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:11.195049 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d9aff-cac3-4fa5-96e7-c05fa5a6f066 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:11.229780 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e4867-abc8-48de-ace7-e97bb2ecd1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:11.230493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e4867-abc8-48de-ace7-e97bb2ecd1f5 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:11.230776 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e4867-abc8-48de-ace7-e97bb2ecd1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:11.231049 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e4867-abc8-48de-ace7-e97bb2ecd1f5 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:11.231820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e4867-abc8-48de-ace7-e97bb2ecd1f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:11.233342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e4867-abc8-48de-ace7-e97bb2ecd1f5 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:11.254087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 44/87] 10.222.0.71 () {60 vars in 1132 bytes} [Thu Jul 21 20:25:11 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:25:11.255137 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e4867-abc8-48de-ace7-e97bb2ecd1f5 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:11.272674 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:11.295762 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece91a22-41bb-4984-b7b2-b9d5cf636cf7 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:11.299896 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:11.340833 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f03e34-35b1-4c19-9f61-ba3605890cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:11.341799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f03e34-35b1-4c19-9f61-ba3605890cc5 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:11.342093 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f03e34-35b1-4c19-9f61-ba3605890cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:11.342750 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f03e34-35b1-4c19-9f61-ba3605890cc5 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:11.343812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f03e34-35b1-4c19-9f61-ba3605890cc5 None admin] RBAC: Adding query filter params (name=member) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:11.344130 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f03e34-35b1-4c19-9f61-ba3605890cc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:11.345643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f03e34-35b1-4c19-9f61-ba3605890cc5 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:11.358566 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 44/88] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:11.359448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f03e34-35b1-4c19-9f61-ba3605890cc5 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:11.388992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e4867-abc8-48de-ace7-e97bb2ecd1f5 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:11.426220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd281ef0-9a64-446e-9c2d-71e4d89c75eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:11.426962 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd281ef0-9a64-446e-9c2d-71e4d89c75eb None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:11.427316 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd281ef0-9a64-446e-9c2d-71e4d89c75eb None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:11.427606 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd281ef0-9a64-446e-9c2d-71e4d89c75eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:11.428477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd281ef0-9a64-446e-9c2d-71e4d89c75eb None admin] RBAC: Adding query filter params (role.id=8a0eeb2a21dc4897a58fb5851d58678c, scope.project.id=3790dd61247c450ebbf3ef2e90a818e0, user.id=89583f5f42ab49058e780d3e5c44662c) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:11.428778 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd281ef0-9a64-446e-9c2d-71e4d89c75eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:11.430318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd281ef0-9a64-446e-9c2d-71e4d89c75eb None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:11.440245 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 45/89] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:25:11 2022] GET /identity/v3/role_assignments?user.id=89583f5f42ab49058e780d3e5c44662c&scope.project.id=3790dd61247c450ebbf3ef2e90a818e0&role.id=8a0eeb2a21dc4897a58fb5851d58678c => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:11.441540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd281ef0-9a64-446e-9c2d-71e4d89c75eb None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:11.503895 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f03e34-35b1-4c19-9f61-ba3605890cc5 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:11.558142 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5ce0b-6b63-406b-b1aa-ed507c2e6b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:11.558919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5ce0b-6b63-406b-b1aa-ed507c2e6b90 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:11.559215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5ce0b-6b63-406b-b1aa-ed507c2e6b90 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:11.559500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5ce0b-6b63-406b-b1aa-ed507c2e6b90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:11.572612 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:11.581445 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:11.583883 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd281ef0-9a64-446e-9c2d-71e4d89c75eb None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:11.628264 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec0074-fcd3-4828-be26-90e1cbd8a880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:11.628934 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec0074-fcd3-4828-be26-90e1cbd8a880 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:11.629275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec0074-fcd3-4828-be26-90e1cbd8a880 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:11.629576 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec0074-fcd3-4828-be26-90e1cbd8a880 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:11.630428 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec0074-fcd3-4828-be26-90e1cbd8a880 None admin] RBAC: Adding query filter params (role.id=19f5e72c0be44a1197c7288de82e79e8, scope.project.id=3790dd61247c450ebbf3ef2e90a818e0, user.id=891d3014859343d8a558b44fef8a2f4f) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:11.630743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec0074-fcd3-4828-be26-90e1cbd8a880 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:11.632236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec0074-fcd3-4828-be26-90e1cbd8a880 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:11.638412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 46/90] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:25:11 2022] GET /identity/v3/role_assignments?user.id=891d3014859343d8a558b44fef8a2f4f&scope.project.id=3790dd61247c450ebbf3ef2e90a818e0&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:11.639290 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec0074-fcd3-4828-be26-90e1cbd8a880 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:11.659012 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5ce0b-6b63-406b-b1aa-ed507c2e6b90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:11.661461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5ce0b-6b63-406b-b1aa-ed507c2e6b90 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:11.781710 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec0074-fcd3-4828-be26-90e1cbd8a880 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:11.814328 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 45/91] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:11.821013 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5ce0b-6b63-406b-b1aa-ed507c2e6b90 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:11.828600 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97739148-b786-45e7-9915-9422615bc04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:11.829647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97739148-b786-45e7-9915-9422615bc04a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:11.830001 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97739148-b786-45e7-9915-9422615bc04a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:11.830356 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97739148-b786-45e7-9915-9422615bc04a None admin] PATH_INFO: `/v3/users/89583f5f42ab49058e780d3e5c44662c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:11.831683 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97739148-b786-45e7-9915-9422615bc04a None admin] RBAC: Authorizing `identity:get_user(user_id=89583f5f42ab49058e780d3e5c44662c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:11.833322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97739148-b786-45e7-9915-9422615bc04a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:11.835242 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 47/92] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:11 2022] GET /identity/v3/users/89583f5f42ab49058e780d3e5c44662c => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:11.842614 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97739148-b786-45e7-9915-9422615bc04a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:11.876557 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:11.900267 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:11.949771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5ce0b-6b63-406b-b1aa-ed507c2e6b90 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:11.997848 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312aafc5-44c4-4ac3-9956-b2395ec1b7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:11.999144 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312aafc5-44c4-4ac3-9956-b2395ec1b7d5 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:11.999651 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312aafc5-44c4-4ac3-9956-b2395ec1b7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:12.000073 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312aafc5-44c4-4ac3-9956-b2395ec1b7d5 None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:12.001517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312aafc5-44c4-4ac3-9956-b2395ec1b7d5 None admin] RBAC: Authorizing `identity:get_project(project_id=3790dd61247c450ebbf3ef2e90a818e0)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:12.003310 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312aafc5-44c4-4ac3-9956-b2395ec1b7d5 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:12.006698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 46/93] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:11 2022] GET /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:12.011830 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312aafc5-44c4-4ac3-9956-b2395ec1b7d5 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:12.082503 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97739148-b786-45e7-9915-9422615bc04a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:12.139203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e77c2-043c-4d4b-a43c-e127c58b3b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:12.140861 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e77c2-043c-4d4b-a43c-e127c58b3b49 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:12.141169 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e77c2-043c-4d4b-a43c-e127c58b3b49 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:12.141454 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e77c2-043c-4d4b-a43c-e127c58b3b49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:12.220905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312aafc5-44c4-4ac3-9956-b2395ec1b7d5 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:12.261282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:12.316772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e77c2-043c-4d4b-a43c-e127c58b3b49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:12.318334 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e77c2-043c-4d4b-a43c-e127c58b3b49 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:12.324691 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa55d4-6ae4-4ba5-8d9c-fa4ee4349159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:12.325936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa55d4-6ae4-4ba5-8d9c-fa4ee4349159 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:12.326275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa55d4-6ae4-4ba5-8d9c-fa4ee4349159 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:12.326559 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa55d4-6ae4-4ba5-8d9c-fa4ee4349159 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:12.327422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa55d4-6ae4-4ba5-8d9c-fa4ee4349159 None admin] RBAC: Adding query filter params (role.id=bad7164dd96e4048bab9360ef3f6769e, scope.project.id=3790dd61247c450ebbf3ef2e90a818e0, user.id=89583f5f42ab49058e780d3e5c44662c) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:12.327740 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa55d4-6ae4-4ba5-8d9c-fa4ee4349159 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:12.329247 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa55d4-6ae4-4ba5-8d9c-fa4ee4349159 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:12.348495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 47/94] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:25:12 2022] GET /identity/v3/role_assignments?user.id=89583f5f42ab49058e780d3e5c44662c&scope.project.id=3790dd61247c450ebbf3ef2e90a818e0&role.id=bad7164dd96e4048bab9360ef3f6769e => generated 259 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:12.361813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:12.505203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 48/95] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:12.511494 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa55d4-6ae4-4ba5-8d9c-fa4ee4349159 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:12.682094 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa55d4-6ae4-4ba5-8d9c-fa4ee4349159 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:12.734706 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d0dcb-0245-4710-91a8-944b103cdd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:12.735594 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d0dcb-0245-4710-91a8-944b103cdd9c None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:12.735880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d0dcb-0245-4710-91a8-944b103cdd9c None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:12.736164 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d0dcb-0245-4710-91a8-944b103cdd9c None admin] PATH_INFO: `/v3/projects/7556db6b05094000a819215f2f3e7f47` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:12.737429 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d0dcb-0245-4710-91a8-944b103cdd9c None admin] RBAC: Authorizing `identity:get_project(project_id=7556db6b05094000a819215f2f3e7f47)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:12.738904 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d0dcb-0245-4710-91a8-944b103cdd9c None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:12.740622 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 48/96] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:12 2022] GET /identity/v3/projects/7556db6b05094000a819215f2f3e7f47 => generated 312 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:12.746644 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e77c2-043c-4d4b-a43c-e127c58b3b49 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:12.894702 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e77c2-043c-4d4b-a43c-e127c58b3b49 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:12.928021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:12.941391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f28755-87a7-41a5-8390-bc457a4e066e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:12.942073 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f28755-87a7-41a5-8390-bc457a4e066e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:12.942365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f28755-87a7-41a5-8390-bc457a4e066e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:12.942642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f28755-87a7-41a5-8390-bc457a4e066e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:12.943501 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f28755-87a7-41a5-8390-bc457a4e066e None admin] RBAC: Adding query filter params (role.id=8a0eeb2a21dc4897a58fb5851d58678c, scope.project.id=7556db6b05094000a819215f2f3e7f47, user.id=89583f5f42ab49058e780d3e5c44662c) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:12.943822 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f28755-87a7-41a5-8390-bc457a4e066e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:12.945324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f28755-87a7-41a5-8390-bc457a4e066e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:12.957605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 49/97] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:25:12 2022] GET /identity/v3/role_assignments?user.id=89583f5f42ab49058e780d3e5c44662c&scope.project.id=7556db6b05094000a819215f2f3e7f47&role.id=8a0eeb2a21dc4897a58fb5851d58678c => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:14.094563 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1abbf65-81c3-46c7-8c7e-5db1b506a4aa None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:14.095195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1abbf65-81c3-46c7-8c7e-5db1b506a4aa None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:14.095195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1abbf65-81c3-46c7-8c7e-5db1b506a4aa None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:14.096654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 49/98] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:25:14.110678 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3858d4-27c0-4ecf-829a-67f9ef7e9fff None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:14.111143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3858d4-27c0-4ecf-829a-67f9ef7e9fff None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:14.111326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3858d4-27c0-4ecf-829a-67f9ef7e9fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:14.134101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-5e3858d4-27c0-4ecf-829a-67f9ef7e9fff None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:14.135654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-5e3858d4-27c0-4ecf-829a-67f9ef7e9fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:14.219752 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 50/99] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:14.263351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1abbf65-81c3-46c7-8c7e-5db1b506a4aa None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:14.332203 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:14.410759 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1abbf65-81c3-46c7-8c7e-5db1b506a4aa None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:14.452874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ffadbd-6538-4273-a609-8551db5738c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:14.453713 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffadbd-6538-4273-a609-8551db5738c4 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:14.454002 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffadbd-6538-4273-a609-8551db5738c4 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:14.454285 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffadbd-6538-4273-a609-8551db5738c4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:14.475308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ffadbd-6538-4273-a609-8551db5738c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:14.481481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-47ffadbd-6538-4273-a609-8551db5738c4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:25:14.481481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:14.481481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:14.481481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:14.481481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:14.481481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:14.481481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:14.481481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:14.481481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:14.481481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:14.481481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:14.481481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:14.481481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:14.481481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:14.481481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:14.481481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:14.482457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:14.482457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:14.482457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:14.482457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:14.482457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:14.482457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:14.482457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:14.482457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:14.482457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:14.482457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:14.482457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:14.482457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:14.482457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:14.482457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:14.482457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:14.482457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:14.483281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:14.483281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:14.483281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:14.483281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:14.483281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:14.483281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:14.483281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:14.483281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:14.483281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:14.483281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:14.483281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:25:14.483281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:14.483281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 50/100] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:25:14.490528 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3858d4-27c0-4ecf-829a-67f9ef7e9fff None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:14.534729 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113a9bf-6dd5-4352-81e4-8b1dca51bc43 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:14.535081 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113a9bf-6dd5-4352-81e4-8b1dca51bc43 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:14.535392 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113a9bf-6dd5-4352-81e4-8b1dca51bc43 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:14.536945 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 51/101] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:25:14.542187 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea34f2c-1739-4884-8f8b-b541e31b5ffb None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:14.542701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea34f2c-1739-4884-8f8b-b541e31b5ffb None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:14.542988 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea34f2c-1739-4884-8f8b-b541e31b5ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:14.582331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-6ea34f2c-1739-4884-8f8b-b541e31b5ffb None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:14.583582 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-6ea34f2c-1739-4884-8f8b-b541e31b5ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:14.643020 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3858d4-27c0-4ecf-829a-67f9ef7e9fff None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:14.671316 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 52/102] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:14.673362 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab5627-1531-4169-ad38-ab5f178140e7 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:14.673875 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab5627-1531-4169-ad38-ab5f178140e7 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:14.674351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab5627-1531-4169-ad38-ab5f178140e7 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:14.676018 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 53/103] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:14.683354 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c770f1-3063-49a1-bb68-60b8f3956519 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:14.683947 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c770f1-3063-49a1-bb68-60b8f3956519 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:14.684434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c770f1-3063-49a1-bb68-60b8f3956519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:14.687324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:14.718373 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-75c770f1-3063-49a1-bb68-60b8f3956519 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:14.720311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-75c770f1-3063-49a1-bb68-60b8f3956519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:14.735805 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b9eb4-54ca-46a4-b5a2-580cde1c678a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:14.736821 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b9eb4-54ca-46a4-b5a2-580cde1c678a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:14.737358 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b9eb4-54ca-46a4-b5a2-580cde1c678a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:14.737896 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b9eb4-54ca-46a4-b5a2-580cde1c678a None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:14.738861 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b9eb4-54ca-46a4-b5a2-580cde1c678a None admin] RBAC: Adding query filter params (name=reader) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:14.739374 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b9eb4-54ca-46a4-b5a2-580cde1c678a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:14.742272 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b9eb4-54ca-46a4-b5a2-580cde1c678a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:14.760387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 51/104] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:14.761148 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b9eb4-54ca-46a4-b5a2-580cde1c678a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:14.767171 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:14.828209 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 54/105] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:14.830040 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64901fd7-767a-4af4-983b-623706cbef4a None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:14.830565 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64901fd7-767a-4af4-983b-623706cbef4a None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:14.831039 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64901fd7-767a-4af4-983b-623706cbef4a None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:14.832711 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 55/106] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:14.833906 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64901fd7-767a-4af4-983b-623706cbef4a None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:14.902422 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b9eb4-54ca-46a4-b5a2-580cde1c678a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:14.949901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5235c2-808c-45d8-91d5-c82a0c1005f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:14.951118 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5235c2-808c-45d8-91d5-c82a0c1005f3 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:14.951596 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5235c2-808c-45d8-91d5-c82a0c1005f3 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:14.952481 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5235c2-808c-45d8-91d5-c82a0c1005f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:14.970438 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5235c2-808c-45d8-91d5-c82a0c1005f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:14.978047 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64901fd7-767a-4af4-983b-623706cbef4a None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:14.982434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-fa5235c2-808c-45d8-91d5-c82a0c1005f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:14.982434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:14.982434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:14.982434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:14.982434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:14.982434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:14.982434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:14.982434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:14.982434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:14.982434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:14.982434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:14.982434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:14.982434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:14.982434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:14.982434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:14.982434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:14.983297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:14.983297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:14.983297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:14.983297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:14.983297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:14.983297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:14.983297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:14.983297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:14.983297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:14.983297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:14.983297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:14.983297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:14.983297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:14.983297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:14.983297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:14.983297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:14.984059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:14.984059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:14.984059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:14.984059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:14.984059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:14.984059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:14.984059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:14.984059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:14.984059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:14.984059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:14.984059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:14.984059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:14.987118 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 52/107] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:14.988601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12cbc63-53e1-4c92-bc71-081fdf588219 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:14.989546 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12cbc63-53e1-4c92-bc71-081fdf588219 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:14.990027 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12cbc63-53e1-4c92-bc71-081fdf588219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:15.013207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-c12cbc63-53e1-4c92-bc71-081fdf588219 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:15.014670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-c12cbc63-53e1-4c92-bc71-081fdf588219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:15.015024 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a741a-0814-4d85-b420-a8a1ae9ea384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:15.015956 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a741a-0814-4d85-b420-a8a1ae9ea384 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:15.016243 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a741a-0814-4d85-b420-a8a1ae9ea384 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:15.016517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a741a-0814-4d85-b420-a8a1ae9ea384 None admin] PATH_INFO: `/v3/users/2042d6d62f0e47fdb382f8a7ab759b9c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:15.033750 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a741a-0814-4d85-b420-a8a1ae9ea384 None admin] RBAC: Authorizing `identity:get_user(user_id=2042d6d62f0e47fdb382f8a7ab759b9c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:15.036234 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a741a-0814-4d85-b420-a8a1ae9ea384 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:15.038124 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 56/108] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:14 2022] GET /identity/v3/users/2042d6d62f0e47fdb382f8a7ab759b9c => generated 289 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:15.038702 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a741a-0814-4d85-b420-a8a1ae9ea384 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:15.058563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:15.064865 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:15.114493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 53/109] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:15.116191 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12cbc63-53e1-4c92-bc71-081fdf588219 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:15.170854 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a741a-0814-4d85-b420-a8a1ae9ea384 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:15.207289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954c770-6ee7-4b00-89c5-3f2982a6dbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:15.208012 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954c770-6ee7-4b00-89c5-3f2982a6dbb6 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:15.208297 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954c770-6ee7-4b00-89c5-3f2982a6dbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:15.208574 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954c770-6ee7-4b00-89c5-3f2982a6dbb6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:15.209897 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954c770-6ee7-4b00-89c5-3f2982a6dbb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:15.211349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954c770-6ee7-4b00-89c5-3f2982a6dbb6 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:15.213179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 57/110] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:25:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:15.218249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954c770-6ee7-4b00-89c5-3f2982a6dbb6 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:15.244595 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12cbc63-53e1-4c92-bc71-081fdf588219 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:15.283538 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e957b-fc66-4075-94a4-6788efe48a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:15.284580 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e957b-fc66-4075-94a4-6788efe48a66 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:15.285091 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e957b-fc66-4075-94a4-6788efe48a66 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:15.285613 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e957b-fc66-4075-94a4-6788efe48a66 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:15.286687 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e957b-fc66-4075-94a4-6788efe48a66 None admin] RBAC: Adding query filter params (name=member) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:15.287206 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e957b-fc66-4075-94a4-6788efe48a66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:15.288852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e957b-fc66-4075-94a4-6788efe48a66 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:15.299213 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 54/111] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:15.300450 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e957b-fc66-4075-94a4-6788efe48a66 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:15.309704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:15.321270 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954c770-6ee7-4b00-89c5-3f2982a6dbb6 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:15.337849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:15.361157 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab0402-3e0a-4e3a-b31a-a7c8b72649ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:15.361951 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab0402-3e0a-4e3a-b31a-a7c8b72649ea None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:15.362254 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab0402-3e0a-4e3a-b31a-a7c8b72649ea None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:15.362580 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab0402-3e0a-4e3a-b31a-a7c8b72649ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:15.417890 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e957b-fc66-4075-94a4-6788efe48a66 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:15.438313 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab0402-3e0a-4e3a-b31a-a7c8b72649ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:15.439882 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab0402-3e0a-4e3a-b31a-a7c8b72649ea None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:15.456817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c369514-78a3-4e95-ac20-7debb5f460f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:15.458108 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c369514-78a3-4e95-ac20-7debb5f460f2 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:15.458615 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c369514-78a3-4e95-ac20-7debb5f460f2 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:15.459248 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c369514-78a3-4e95-ac20-7debb5f460f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:15.473463 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c369514-78a3-4e95-ac20-7debb5f460f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:15.482820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-8c369514-78a3-4e95-ac20-7debb5f460f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:15.482820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:15.482820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:15.482820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:15.482820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:15.482820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:15.482820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:15.482820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:15.482820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:15.482820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:15.482820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:15.482820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:15.482820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:15.482820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:15.482820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:15.482820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:15.483764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:15.483764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:15.483764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:15.483764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:15.483764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:15.483764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:15.483764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:15.483764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:15.483764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:15.483764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:15.483764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:15.483764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:15.483764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:15.483764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:15.483764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:15.483764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:15.484543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:15.484543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:15.484543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:15.484543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:15.484543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:15.484543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:15.484543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:15.484543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:15.484543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:15.484543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:15.484543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:15.484543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:15.487333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 55/112] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:25:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:15.488310 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c369514-78a3-4e95-ac20-7debb5f460f2 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:15.545048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:15.558302 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 58/113] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:15.559166 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab0402-3e0a-4e3a-b31a-a7c8b72649ea None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:15.619192 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:15.632007 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c369514-78a3-4e95-ac20-7debb5f460f2 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:15.666452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab0402-3e0a-4e3a-b31a-a7c8b72649ea None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:15.670874 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff13268-913d-4c17-a09e-c30fbeb02519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:15.671804 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff13268-913d-4c17-a09e-c30fbeb02519 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:15.672295 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff13268-913d-4c17-a09e-c30fbeb02519 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:15.672775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff13268-913d-4c17-a09e-c30fbeb02519 None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:15.673738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff13268-913d-4c17-a09e-c30fbeb02519 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:15.675532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff13268-913d-4c17-a09e-c30fbeb02519 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:15.703788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81ed2f-b89e-4735-8487-d1a11576a9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:15.704725 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81ed2f-b89e-4735-8487-d1a11576a9c5 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:15.704838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 56/114] 10.222.0.71 () {60 vars in 1141 bytes} [Thu Jul 21 20:25:15 2022] POST /identity/v3/projects => generated 302 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:25:15.705318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81ed2f-b89e-4735-8487-d1a11576a9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:15.705318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81ed2f-b89e-4735-8487-d1a11576a9c5 None admin] PATH_INFO: `/v3/users/89583f5f42ab49058e780d3e5c44662c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:15.706532 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81ed2f-b89e-4735-8487-d1a11576a9c5 None admin] RBAC: Authorizing `identity:get_user(user_id=89583f5f42ab49058e780d3e5c44662c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:15.708297 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81ed2f-b89e-4735-8487-d1a11576a9c5 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:15.709927 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff13268-913d-4c17-a09e-c30fbeb02519 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:15.714962 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 59/115] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:15 2022] GET /identity/v3/users/89583f5f42ab49058e780d3e5c44662c => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:15.714962 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81ed2f-b89e-4735-8487-d1a11576a9c5 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:15.803419 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:15.834170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff13268-913d-4c17-a09e-c30fbeb02519 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:15.853797 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81ed2f-b89e-4735-8487-d1a11576a9c5 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:15.878187 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9bb505-8049-4603-a95c-bf06ef26abc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:15.878866 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9bb505-8049-4603-a95c-bf06ef26abc6 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:15.879146 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9bb505-8049-4603-a95c-bf06ef26abc6 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:15.879422 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9bb505-8049-4603-a95c-bf06ef26abc6 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:15.880177 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9bb505-8049-4603-a95c-bf06ef26abc6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:15.880484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9bb505-8049-4603-a95c-bf06ef26abc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:15.881911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9bb505-8049-4603-a95c-bf06ef26abc6 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:15.891077 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 57/116] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:15.892347 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d86bf-99f9-441d-a034-f54bf6162d14 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:15.892638 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d86bf-99f9-441d-a034-f54bf6162d14 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:15.892912 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d86bf-99f9-441d-a034-f54bf6162d14 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:15.894185 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 58/117] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:15.894872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d86bf-99f9-441d-a034-f54bf6162d14 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:15.898408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:15.900264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248458fc-fc80-451c-8d45-1b6d554dbffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:15.901470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248458fc-fc80-451c-8d45-1b6d554dbffe None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:15.902813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248458fc-fc80-451c-8d45-1b6d554dbffe None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:15.902813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248458fc-fc80-451c-8d45-1b6d554dbffe None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:15.903306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248458fc-fc80-451c-8d45-1b6d554dbffe None admin] RBAC: Authorizing `identity:get_project(project_id=3790dd61247c450ebbf3ef2e90a818e0)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:15.905723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248458fc-fc80-451c-8d45-1b6d554dbffe None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:15.907745 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 60/118] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:15 2022] GET /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:15.908774 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887346ec-4145-43e8-a64f-eebe8886cd25 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:15.909598 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887346ec-4145-43e8-a64f-eebe8886cd25 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:15.909598 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887346ec-4145-43e8-a64f-eebe8886cd25 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:15.910742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 61/119] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:15.911684 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef93ab-675e-4c64-bb9f-817a0efa94a2 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:15.911976 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef93ab-675e-4c64-bb9f-817a0efa94a2 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:15.912256 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef93ab-675e-4c64-bb9f-817a0efa94a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:15.931970 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-7cef93ab-675e-4c64-bb9f-817a0efa94a2 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:15.933257 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-7cef93ab-675e-4c64-bb9f-817a0efa94a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:16.007826 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 62/120] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:16.009273 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef93ab-675e-4c64-bb9f-817a0efa94a2 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:16.033699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d86bf-99f9-441d-a034-f54bf6162d14 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:16.071208 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08db004-7508-48c0-a5a5-4af07269cce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:16.072463 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08db004-7508-48c0-a5a5-4af07269cce5 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:16.072965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08db004-7508-48c0-a5a5-4af07269cce5 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:16.073492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08db004-7508-48c0-a5a5-4af07269cce5 None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:16.074998 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08db004-7508-48c0-a5a5-4af07269cce5 None admin] RBAC: Authorizing `identity:get_project(project_id=3790dd61247c450ebbf3ef2e90a818e0)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:16.077286 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08db004-7508-48c0-a5a5-4af07269cce5 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:16.079306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 59/121] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:15 2022] GET /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:16.084545 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:16.086702 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260cd8d-e145-4b3a-803d-46971980b61f None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:16.087008 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260cd8d-e145-4b3a-803d-46971980b61f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:16.087281 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260cd8d-e145-4b3a-803d-46971980b61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:16.105761 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-8260cd8d-e145-4b3a-803d-46971980b61f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:16.107042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-8260cd8d-e145-4b3a-803d-46971980b61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:16.131494 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef93ab-675e-4c64-bb9f-817a0efa94a2 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:16.168856 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9c1dd-cd04-442f-8749-68087a090741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:16.169895 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9c1dd-cd04-442f-8749-68087a090741 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:16.170190 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9c1dd-cd04-442f-8749-68087a090741 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:16.170470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9c1dd-cd04-442f-8749-68087a090741 None admin] PATH_INFO: `/v3/users/891d3014859343d8a558b44fef8a2f4f` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:16.172027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e9c1dd-cd04-442f-8749-68087a090741 None admin] RBAC: Authorizing `identity:get_user(user_id=891d3014859343d8a558b44fef8a2f4f)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:16.173836 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e9c1dd-cd04-442f-8749-68087a090741 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:16.175716 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 63/122] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:16 2022] GET /identity/v3/users/891d3014859343d8a558b44fef8a2f4f => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:16.180881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9c1dd-cd04-442f-8749-68087a090741 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:16.185946 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 60/123] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:16.187328 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260cd8d-e145-4b3a-803d-46971980b61f None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:16.190067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:16.287404 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9c1dd-cd04-442f-8749-68087a090741 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:16.299742 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260cd8d-e145-4b3a-803d-46971980b61f None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:16.313105 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:16.326470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456bf86-9b7a-4023-b569-3ed322694ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:16.327357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456bf86-9b7a-4023-b569-3ed322694ba6 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:16.327803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456bf86-9b7a-4023-b569-3ed322694ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:16.328240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456bf86-9b7a-4023-b569-3ed322694ba6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:16.329268 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456bf86-9b7a-4023-b569-3ed322694ba6 None admin] RBAC: Adding query filter params (role.id=bc82ed55215d47b791d5aa25699e7e2b, scope.project.id=3790dd61247c450ebbf3ef2e90a818e0, user.id=2042d6d62f0e47fdb382f8a7ab759b9c) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:16.329763 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456bf86-9b7a-4023-b569-3ed322694ba6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:16.331243 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456bf86-9b7a-4023-b569-3ed322694ba6 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:16.337469 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafd59c-db20-47c6-bbaf-d073bc544b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:16.337723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 64/124] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:25:16 2022] GET /identity/v3/role_assignments?user.id=2042d6d62f0e47fdb382f8a7ab759b9c&scope.project.id=3790dd61247c450ebbf3ef2e90a818e0&role.id=bc82ed55215d47b791d5aa25699e7e2b => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:16.338573 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafd59c-db20-47c6-bbaf-d073bc544b6d None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:16.338954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafd59c-db20-47c6-bbaf-d073bc544b6d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:16.339236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafd59c-db20-47c6-bbaf-d073bc544b6d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:16.342451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456bf86-9b7a-4023-b569-3ed322694ba6 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:16.362811 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafd59c-db20-47c6-bbaf-d073bc544b6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:16.371186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-5bafd59c-db20-47c6-bbaf-d073bc544b6d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:16.371186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:16.371186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:16.371186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:16.371186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:16.371186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:16.371186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:16.371186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:16.371186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:16.371186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:16.371186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:16.371186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:16.371186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:16.371186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:16.371186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:16.371186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:16.372143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:16.372143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:16.372143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:16.372143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:16.372143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:16.372143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:16.372143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:16.372143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:16.372143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:16.372143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:16.372143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:16.372143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:16.372143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:16.372143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:16.372143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:16.372143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:16.372936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:16.372936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:16.372936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:16.372936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:16.372936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:16.372936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:16.372936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:16.372936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:16.372936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:16.372936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:16.372936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:16.372936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:16.372936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 61/125] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:25:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:16.374120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafd59c-db20-47c6-bbaf-d073bc544b6d None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:16.444927 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:16.504315 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456bf86-9b7a-4023-b569-3ed322694ba6 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:16.509750 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafd59c-db20-47c6-bbaf-d073bc544b6d None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:16.543743 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de6468-ddb7-4292-81c3-6bc6a7444cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:16.545041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de6468-ddb7-4292-81c3-6bc6a7444cd5 None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:16.545491 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de6468-ddb7-4292-81c3-6bc6a7444cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:16.545977 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de6468-ddb7-4292-81c3-6bc6a7444cd5 None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0/users/89583f5f42ab49058e780d3e5c44662c/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:16.548203 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de6468-ddb7-4292-81c3-6bc6a7444cd5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3790dd61247c450ebbf3ef2e90a818e0, user_id=89583f5f42ab49058e780d3e5c44662c, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:16.549973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de6468-ddb7-4292-81c3-6bc6a7444cd5 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:16.555124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459de671-af2a-4072-a20b-c6aaeb06b927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:16.556405 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459de671-af2a-4072-a20b-c6aaeb06b927 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:16.556519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459de671-af2a-4072-a20b-c6aaeb06b927 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:16.556913 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459de671-af2a-4072-a20b-c6aaeb06b927 None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:16.558617 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459de671-af2a-4072-a20b-c6aaeb06b927 None admin] RBAC: Authorizing `identity:get_project(project_id=3790dd61247c450ebbf3ef2e90a818e0)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:16.560542 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459de671-af2a-4072-a20b-c6aaeb06b927 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:16.562465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 62/126] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:16 2022] GET /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:16.568320 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459de671-af2a-4072-a20b-c6aaeb06b927 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:16.570716 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 65/127] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:25:16 2022] PUT /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0/users/89583f5f42ab49058e780d3e5c44662c/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:25:16.572008 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de6468-ddb7-4292-81c3-6bc6a7444cd5 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:16.629955 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:16.692069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459de671-af2a-4072-a20b-c6aaeb06b927 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:16.727744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de6468-ddb7-4292-81c3-6bc6a7444cd5 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:16.734865 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:16.741751 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b434b-06c9-4fb7-bf7d-7a964be5046d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:16.743034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b434b-06c9-4fb7-bf7d-7a964be5046d None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:16.743568 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b434b-06c9-4fb7-bf7d-7a964be5046d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:16.744060 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b434b-06c9-4fb7-bf7d-7a964be5046d None admin] PATH_INFO: `/v3/roles/member` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:16.774885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b434b-06c9-4fb7-bf7d-7a964be5046d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:16.790042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-470b434b-06c9-4fb7-bf7d-7a964be5046d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:16.790042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:16.790042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:16.790042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:16.790042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:16.790042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:16.790042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:16.790042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:16.790042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:16.790042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:16.790042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:16.790042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:16.790042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:16.790042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:16.790042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:16.790042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:16.791231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:16.791231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:16.791231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:16.791231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:16.791231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:16.791231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:16.791231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:16.791231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:16.791231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:16.791231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:16.791231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:16.791231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:16.791231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:16.791231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:16.791231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:16.791231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:16.792301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:16.792301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:16.792301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:16.792301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:16.792301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:16.792301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:16.792301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:16.792301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:16.792301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:16.792301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:16.792301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:16.792301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:16.792301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 63/128] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:16.792926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b434b-06c9-4fb7-bf7d-7a964be5046d None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:16.810302 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ea90f-feac-48e2-b253-821cb8e8d7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:16.811871 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ea90f-feac-48e2-b253-821cb8e8d7c5 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:16.812736 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ea90f-feac-48e2-b253-821cb8e8d7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:16.813219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ea90f-feac-48e2-b253-821cb8e8d7c5 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:16.814477 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ea90f-feac-48e2-b253-821cb8e8d7c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:16.814976 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ea90f-feac-48e2-b253-821cb8e8d7c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:16.817346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ea90f-feac-48e2-b253-821cb8e8d7c5 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:16.843769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 66/129] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:25:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:16.844663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ea90f-feac-48e2-b253-821cb8e8d7c5 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:16.935847 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b434b-06c9-4fb7-bf7d-7a964be5046d None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:16.969085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ea90f-feac-48e2-b253-821cb8e8d7c5 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:16.975784 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff35f5-1d73-41ef-b173-1353836c8917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:16.976743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff35f5-1d73-41ef-b173-1353836c8917 None admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:16.977030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff35f5-1d73-41ef-b173-1353836c8917 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:16.977375 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff35f5-1d73-41ef-b173-1353836c8917 None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0/users/891d3014859343d8a558b44fef8a2f4f/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:16.979411 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff35f5-1d73-41ef-b173-1353836c8917 None admin] RBAC: Authorizing `identity:create_grant(project_id=3790dd61247c450ebbf3ef2e90a818e0, user_id=891d3014859343d8a558b44fef8a2f4f, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:16.980895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff35f5-1d73-41ef-b173-1353836c8917 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:17.009100 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:17.009799 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 64/130] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:25:16 2022] PUT /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0/users/891d3014859343d8a558b44fef8a2f4f/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:25:17.010530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff35f5-1d73-41ef-b173-1353836c8917 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:17.022168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8c884-ceb4-4d28-b636-2e1aeaece8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:17.023269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8c884-ceb4-4d28-b636-2e1aeaece8bb None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:17.023748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8c884-ceb4-4d28-b636-2e1aeaece8bb None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:17.024289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8c884-ceb4-4d28-b636-2e1aeaece8bb None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:17.025305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe8c884-ceb4-4d28-b636-2e1aeaece8bb None admin] RBAC: Adding query filter params (name=member) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:17.025814 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe8c884-ceb4-4d28-b636-2e1aeaece8bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:17.027526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe8c884-ceb4-4d28-b636-2e1aeaece8bb None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:17.049106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 67/131] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:17.056229 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8c884-ceb4-4d28-b636-2e1aeaece8bb None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:17.104686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:17.245213 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff35f5-1d73-41ef-b173-1353836c8917 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:17.297050 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43effdd-e900-4bb5-a5f5-f1b01ea796b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:17.298421 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43effdd-e900-4bb5-a5f5-f1b01ea796b8 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:17.298705 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43effdd-e900-4bb5-a5f5-f1b01ea796b8 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:17.298976 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43effdd-e900-4bb5-a5f5-f1b01ea796b8 None admin] PATH_INFO: `/v3/users/89583f5f42ab49058e780d3e5c44662c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:17.300289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43effdd-e900-4bb5-a5f5-f1b01ea796b8 None admin] RBAC: Authorizing `identity:get_user(user_id=89583f5f42ab49058e780d3e5c44662c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:17.302044 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43effdd-e900-4bb5-a5f5-f1b01ea796b8 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:17.304026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 65/132] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:17 2022] GET /identity/v3/users/89583f5f42ab49058e780d3e5c44662c => generated 275 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:17.314691 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43effdd-e900-4bb5-a5f5-f1b01ea796b8 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:17.327530 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8c884-ceb4-4d28-b636-2e1aeaece8bb None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:17.406017 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0baf8e8-e6f1-46c5-b2a9-1b8d7d62938c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:17.406917 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0baf8e8-e6f1-46c5-b2a9-1b8d7d62938c None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:17.407198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0baf8e8-e6f1-46c5-b2a9-1b8d7d62938c None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:17.407471 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0baf8e8-e6f1-46c5-b2a9-1b8d7d62938c None admin] PATH_INFO: `/v3/users/89583f5f42ab49058e780d3e5c44662c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:17.408711 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0baf8e8-e6f1-46c5-b2a9-1b8d7d62938c None admin] RBAC: Authorizing `identity:get_user(user_id=89583f5f42ab49058e780d3e5c44662c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:17.410214 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0baf8e8-e6f1-46c5-b2a9-1b8d7d62938c None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:17.411851 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 68/133] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:17 2022] GET /identity/v3/users/89583f5f42ab49058e780d3e5c44662c => generated 275 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:17.422069 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0baf8e8-e6f1-46c5-b2a9-1b8d7d62938c None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:17.482761 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43effdd-e900-4bb5-a5f5-f1b01ea796b8 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:17.505467 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:17.517468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:17.570714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4ae61-ff77-4236-b589-00f7c9425dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:17.571986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4ae61-ff77-4236-b589-00f7c9425dbf None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:17.572469 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4ae61-ff77-4236-b589-00f7c9425dbf None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:17.572958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4ae61-ff77-4236-b589-00f7c9425dbf None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:17.577838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4ae61-ff77-4236-b589-00f7c9425dbf None admin] RBAC: Authorizing `identity:get_project(project_id=3790dd61247c450ebbf3ef2e90a818e0)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:17.579564 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4ae61-ff77-4236-b589-00f7c9425dbf None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:17.581522 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 66/134] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:17 2022] GET /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:17.587811 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4ae61-ff77-4236-b589-00f7c9425dbf None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:17.668945 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0baf8e8-e6f1-46c5-b2a9-1b8d7d62938c None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:17.741200 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363a81e-f8e3-4b6b-8d3a-c3bf44e39511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:17.742545 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363a81e-f8e3-4b6b-8d3a-c3bf44e39511 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:17.744399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363a81e-f8e3-4b6b-8d3a-c3bf44e39511 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:17.746566 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363a81e-f8e3-4b6b-8d3a-c3bf44e39511 None admin] PATH_INFO: `/v3/projects/7556db6b05094000a819215f2f3e7f47` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:17.751296 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363a81e-f8e3-4b6b-8d3a-c3bf44e39511 None admin] RBAC: Authorizing `identity:get_project(project_id=7556db6b05094000a819215f2f3e7f47)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:17.751296 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363a81e-f8e3-4b6b-8d3a-c3bf44e39511 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:17.751296 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 69/135] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:17 2022] GET /identity/v3/projects/7556db6b05094000a819215f2f3e7f47 => generated 312 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:17.762593 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363a81e-f8e3-4b6b-8d3a-c3bf44e39511 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:17.802501 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4ae61-ff77-4236-b589-00f7c9425dbf None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:17.847711 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a71834-370a-46dd-9248-a3ecc73f683e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:17.849134 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a71834-370a-46dd-9248-a3ecc73f683e None admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:17.853428 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a71834-370a-46dd-9248-a3ecc73f683e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:17.853860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a71834-370a-46dd-9248-a3ecc73f683e None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0/users/89583f5f42ab49058e780d3e5c44662c/roles/bad7164dd96e4048bab9360ef3f6769e` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:17.860664 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a71834-370a-46dd-9248-a3ecc73f683e None admin] RBAC: Authorizing `identity:create_grant(project_id=3790dd61247c450ebbf3ef2e90a818e0, user_id=89583f5f42ab49058e780d3e5c44662c, role_id=bad7164dd96e4048bab9360ef3f6769e)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:17.862452 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a71834-370a-46dd-9248-a3ecc73f683e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:17.888180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 67/136] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:25:17 2022] PUT /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0/users/89583f5f42ab49058e780d3e5c44662c/roles/bad7164dd96e4048bab9360ef3f6769e => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:25:17.945302 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363a81e-f8e3-4b6b-8d3a-c3bf44e39511 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:17.955817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:17.984295 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803eb7d-1a52-4458-bbee-9073b3133c6e None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:17.984970 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803eb7d-1a52-4458-bbee-9073b3133c6e None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:17.985482 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803eb7d-1a52-4458-bbee-9073b3133c6e None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:17.987105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 68/137] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:25:17.992700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e08ad0-b975-4290-b36f-7eea213d7b47 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:17.993331 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e08ad0-b975-4290-b36f-7eea213d7b47 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:17.993872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e08ad0-b975-4290-b36f-7eea213d7b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:17.994371 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7807b-c21f-48ba-8ca0-ffcc05024af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:17.995281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7807b-c21f-48ba-8ca0-ffcc05024af1 None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:17.995570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7807b-c21f-48ba-8ca0-ffcc05024af1 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:17.995852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7807b-c21f-48ba-8ca0-ffcc05024af1 None admin] PATH_INFO: `/v3/projects/7556db6b05094000a819215f2f3e7f47/users/89583f5f42ab49058e780d3e5c44662c/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:17.998537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7807b-c21f-48ba-8ca0-ffcc05024af1 None admin] RBAC: Authorizing `identity:create_grant(project_id=7556db6b05094000a819215f2f3e7f47, user_id=89583f5f42ab49058e780d3e5c44662c, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:18.000047 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7807b-c21f-48ba-8ca0-ffcc05024af1 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:18.017530 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 70/138] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:25:17 2022] PUT /identity/v3/projects/7556db6b05094000a819215f2f3e7f47/users/89583f5f42ab49058e780d3e5c44662c/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:25:18.026240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-15e08ad0-b975-4290-b36f-7eea213d7b47 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:18.027848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-15e08ad0-b975-4290-b36f-7eea213d7b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:18.105269 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:18.139270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 69/139] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:18.176397 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7807b-c21f-48ba-8ca0-ffcc05024af1 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:18.290214 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7807b-c21f-48ba-8ca0-ffcc05024af1 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:18.326449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba26d5a-ca1f-4eca-ab95-20940819dbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:18.327622 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba26d5a-ca1f-4eca-ab95-20940819dbaf None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:18.328520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba26d5a-ca1f-4eca-ab95-20940819dbaf None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:18.328992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba26d5a-ca1f-4eca-ab95-20940819dbaf None admin] PATH_INFO: `/v3/domains/default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:18.330405 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba26d5a-ca1f-4eca-ab95-20940819dbaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:18.332066 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba26d5a-ca1f-4eca-ab95-20940819dbaf None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:18.334236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 71/140] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:25:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:18.339531 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e08ad0-b975-4290-b36f-7eea213d7b47 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:18.502047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e08ad0-b975-4290-b36f-7eea213d7b47 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:18.558135 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d237c-1be1-4d1e-8fff-27293b7c390b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:18.558744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d237c-1be1-4d1e-8fff-27293b7c390b None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:18.559025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d237c-1be1-4d1e-8fff-27293b7c390b None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:18.559301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d237c-1be1-4d1e-8fff-27293b7c390b None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:18.560578 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d237c-1be1-4d1e-8fff-27293b7c390b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:18.562081 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d237c-1be1-4d1e-8fff-27293b7c390b None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:18.593815 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 70/141] 10.222.0.71 () {60 vars in 1132 bytes} [Thu Jul 21 20:25:18 2022] POST /identity/v3/users => generated 283 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:25:19.188973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8175bec-6029-49c6-b425-578f3ee22b3a None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:19.188973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8175bec-6029-49c6-b425-578f3ee22b3a None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:19.188973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8175bec-6029-49c6-b425-578f3ee22b3a None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:19.188973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 72/142] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:19.195992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741f2eb-776e-4d07-908b-32d7ce45dff2 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:19.196591 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741f2eb-776e-4d07-908b-32d7ce45dff2 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:19.197105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741f2eb-776e-4d07-908b-32d7ce45dff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:19.219058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-2741f2eb-776e-4d07-908b-32d7ce45dff2 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-2741f2eb-776e-4d07-908b-32d7ce45dff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:19.262023 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:19.322550 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 71/143] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:19.358898 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8175bec-6029-49c6-b425-578f3ee22b3a None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:19.409167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:19.498237 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8175bec-6029-49c6-b425-578f3ee22b3a None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:19.550045 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b435f-9137-45e6-9a16-a010a95fd708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:19.550864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b435f-9137-45e6-9a16-a010a95fd708 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:19.551150 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b435f-9137-45e6-9a16-a010a95fd708 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:19.551444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b435f-9137-45e6-9a16-a010a95fd708 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:19.569036 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b435f-9137-45e6-9a16-a010a95fd708 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:19.577132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-ba6b435f-9137-45e6-9a16-a010a95fd708 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:25:19.577132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:19.577132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:19.577132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:19.577132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:19.577132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:19.577132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:19.577132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:19.577132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:19.577132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:19.577132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:19.577132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:19.577132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:19.577132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:19.577132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:19.577132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:19.578106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:19.578106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:19.578106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:19.578106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:19.578106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:19.578106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:19.578106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:19.578106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:19.578106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:19.578106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:19.578106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:19.578106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:19.578106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:19.578106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:19.578106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:19.578106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:19.578877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:19.578877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:19.578877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:19.578877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:19.578877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:19.578877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:19.578877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:19.578877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:19.578877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:19.578877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:19.578877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:25:19.578877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:19.578877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 73/144] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:19.584819 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741f2eb-776e-4d07-908b-32d7ce45dff2 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:19.723108 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741f2eb-776e-4d07-908b-32d7ce45dff2 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:19.761419 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6664b1a-986a-4ac3-b8a9-64a22d6683ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:19.762482 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6664b1a-986a-4ac3-b8a9-64a22d6683ff None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:19.763508 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6664b1a-986a-4ac3-b8a9-64a22d6683ff None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:19.763985 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6664b1a-986a-4ac3-b8a9-64a22d6683ff None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:19.764959 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6664b1a-986a-4ac3-b8a9-64a22d6683ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:19.765490 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6664b1a-986a-4ac3-b8a9-64a22d6683ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:19.767128 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6664b1a-986a-4ac3-b8a9-64a22d6683ff None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:19.776865 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 72/145] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:19.782755 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b435f-9137-45e6-9a16-a010a95fd708 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:19.914458 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2483ea3-78ba-4487-830d-d94a19530356 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:19.915257 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2483ea3-78ba-4487-830d-d94a19530356 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:19.916233 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:19.916678 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2483ea3-78ba-4487-830d-d94a19530356 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:19.923438 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 73/146] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:19 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:19.926135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b435f-9137-45e6-9a16-a010a95fd708 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:19.930519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fea57c-22ea-44f0-8ad4-b20a183c6a32 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:19.931107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fea57c-22ea-44f0-8ad4-b20a183c6a32 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:19.931624 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fea57c-22ea-44f0-8ad4-b20a183c6a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:19.962433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-91fea57c-22ea-44f0-8ad4-b20a183c6a32 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:19.964755 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-91fea57c-22ea-44f0-8ad4-b20a183c6a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:19.993608 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:19.995458 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b39bb8-36e2-43aa-9ede-c50893c087ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:19.996372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b39bb8-36e2-43aa-9ede-c50893c087ea None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:19.996655 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b39bb8-36e2-43aa-9ede-c50893c087ea None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:19.996930 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b39bb8-36e2-43aa-9ede-c50893c087ea None admin] PATH_INFO: `/v3/users/2042d6d62f0e47fdb382f8a7ab759b9c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:19.998747 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b39bb8-36e2-43aa-9ede-c50893c087ea None admin] RBAC: Authorizing `identity:get_user(user_id=2042d6d62f0e47fdb382f8a7ab759b9c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:20.000680 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b39bb8-36e2-43aa-9ede-c50893c087ea None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:20.009501 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 74/147] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:19 2022] GET /identity/v3/users/2042d6d62f0e47fdb382f8a7ab759b9c => generated 289 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:20.010467 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b39bb8-36e2-43aa-9ede-c50893c087ea None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:20.069532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 74/148] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:20.082256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcf8d9-0e11-48f8-824c-c57e322fe8da None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:20.082555 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcf8d9-0e11-48f8-824c-c57e322fe8da None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:20.082817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcf8d9-0e11-48f8-824c-c57e322fe8da None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:20.084401 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 75/149] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:25:20.090822 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff1fc9-7514-45d2-b6cd-bfba713428c7 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:20.091840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff1fc9-7514-45d2-b6cd-bfba713428c7 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:20.092135 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff1fc9-7514-45d2-b6cd-bfba713428c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:20.122381 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-96ff1fc9-7514-45d2-b6cd-bfba713428c7 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:20.123600 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-96ff1fc9-7514-45d2-b6cd-bfba713428c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:20.232563 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b39bb8-36e2-43aa-9ede-c50893c087ea None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:20.254037 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 76/150] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:20.254874 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff1fc9-7514-45d2-b6cd-bfba713428c7 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:20.306309 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11007516-1d36-4cee-8ed0-6a176c4dce78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:20.307209 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11007516-1d36-4cee-8ed0-6a176c4dce78 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:20.307495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11007516-1d36-4cee-8ed0-6a176c4dce78 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:20.307773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11007516-1d36-4cee-8ed0-6a176c4dce78 None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:20.309223 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11007516-1d36-4cee-8ed0-6a176c4dce78 None admin] RBAC: Authorizing `identity:get_project(project_id=3790dd61247c450ebbf3ef2e90a818e0)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:20.311312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11007516-1d36-4cee-8ed0-6a176c4dce78 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:20.313539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 75/151] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:20 2022] GET /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0 => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:20.329537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11007516-1d36-4cee-8ed0-6a176c4dce78 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:20.379431 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:20.387210 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:20.402892 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff1fc9-7514-45d2-b6cd-bfba713428c7 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:20.444761 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19495f3-9eac-497f-9427-ce95c19d782c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:20.445673 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19495f3-9eac-497f-9427-ce95c19d782c None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:20.445954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19495f3-9eac-497f-9427-ce95c19d782c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:20.446227 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19495f3-9eac-497f-9427-ce95c19d782c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:20.465896 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19495f3-9eac-497f-9427-ce95c19d782c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:20.480468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-e19495f3-9eac-497f-9427-ce95c19d782c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:20.480468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:20.480468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:20.480468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:20.480468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:20.480468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:20.480468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:20.480468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:20.480468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:20.480468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:20.480468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:20.480468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:20.480468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:20.480468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:20.480468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:20.480468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:20.481492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:20.481492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:20.481492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:20.481492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:20.481492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:20.481492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:20.481492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:20.481492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:20.481492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:20.481492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:20.481492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:20.481492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:20.481492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:20.481492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:20.481492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:20.481492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:20.483621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:20.483621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:20.483621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:20.483621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:20.483621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:20.483621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:20.483621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:20.483621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:20.483621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:20.483621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:20.483621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:20.483621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:20.483621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 77/152] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:25:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:20.483621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19495f3-9eac-497f-9427-ce95c19d782c None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:20.548962 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11007516-1d36-4cee-8ed0-6a176c4dce78 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:20.586169 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe581c8-fae5-4b50-b232-0986d261ac74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:20.587070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe581c8-fae5-4b50-b232-0986d261ac74 None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:20.587451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe581c8-fae5-4b50-b232-0986d261ac74 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:20.587730 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe581c8-fae5-4b50-b232-0986d261ac74 None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0/users/2042d6d62f0e47fdb382f8a7ab759b9c/roles/bc82ed55215d47b791d5aa25699e7e2b` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:20.593912 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe581c8-fae5-4b50-b232-0986d261ac74 None admin] RBAC: Authorizing `identity:create_grant(project_id=3790dd61247c450ebbf3ef2e90a818e0, user_id=2042d6d62f0e47fdb382f8a7ab759b9c, role_id=bc82ed55215d47b791d5aa25699e7e2b)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:20.595386 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe581c8-fae5-4b50-b232-0986d261ac74 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:20.621854 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 76/153] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:25:20 2022] PUT /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0/users/2042d6d62f0e47fdb382f8a7ab759b9c/roles/bc82ed55215d47b791d5aa25699e7e2b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:25:20.622763 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe581c8-fae5-4b50-b232-0986d261ac74 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:20.634191 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19495f3-9eac-497f-9427-ce95c19d782c None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:20.672446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2de50-6616-4d8f-aff7-5423952cdb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:20.673302 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2de50-6616-4d8f-aff7-5423952cdb02 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:20.673717 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2de50-6616-4d8f-aff7-5423952cdb02 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:20.673884 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2de50-6616-4d8f-aff7-5423952cdb02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:20.694586 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2de50-6616-4d8f-aff7-5423952cdb02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:20.703487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-eab2de50-6616-4d8f-aff7-5423952cdb02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:20.703487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:20.703487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:20.703487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:20.703487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:20.703487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:20.703487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:20.703487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:20.703487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:20.703487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:20.703487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:20.703487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:20.703487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:20.703487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:20.703487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:20.703487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:20.705655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:20.705655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:20.705655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:20.705655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:20.705655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:20.705655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:20.705655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:20.705655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:20.705655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:20.705655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:20.705655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:20.705655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:20.705655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:20.705655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:20.705655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:20.705655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:20.707788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:20.707788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:20.707788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:20.707788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:20.707788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:20.707788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:20.707788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:20.707788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:20.707788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:20.707788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:20.707788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:20.707788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:20.707788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 78/154] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:20.715649 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2de50-6616-4d8f-aff7-5423952cdb02 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:20.721429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:20.840926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2de50-6616-4d8f-aff7-5423952cdb02 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:20.881392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc347e-e2f0-47df-a9a5-d52392ce0e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:20.882080 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc347e-e2f0-47df-a9a5-d52392ce0e5c None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:20.882369 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc347e-e2f0-47df-a9a5-d52392ce0e5c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:20.882652 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc347e-e2f0-47df-a9a5-d52392ce0e5c None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:20.883930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc347e-e2f0-47df-a9a5-d52392ce0e5c None admin] RBAC: Adding query filter params (name=member) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:20.884250 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc347e-e2f0-47df-a9a5-d52392ce0e5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:20.886003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc347e-e2f0-47df-a9a5-d52392ce0e5c None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:20.897190 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 79/155] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:20.906544 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc347e-e2f0-47df-a9a5-d52392ce0e5c None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:20.942351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:20.975096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe581c8-fae5-4b50-b232-0986d261ac74 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:21.014049 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:21.041755 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6980b-9227-4d42-8a85-61c68affe321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:21.042448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6980b-9227-4d42-8a85-61c68affe321 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:21.042760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6980b-9227-4d42-8a85-61c68affe321 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:21.043001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6980b-9227-4d42-8a85-61c68affe321 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:21.043785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6980b-9227-4d42-8a85-61c68affe321 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:21.044092 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6980b-9227-4d42-8a85-61c68affe321 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:21.045605 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6980b-9227-4d42-8a85-61c68affe321 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:21.063814 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 77/156] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:21.068508 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6980b-9227-4d42-8a85-61c68affe321 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:21.086462 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc347e-e2f0-47df-a9a5-d52392ce0e5c None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:21.164752 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a787d37-a04d-4c98-b500-a257be81800c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:21.165687 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a787d37-a04d-4c98-b500-a257be81800c None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:21.165968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a787d37-a04d-4c98-b500-a257be81800c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:21.166252 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a787d37-a04d-4c98-b500-a257be81800c None admin] PATH_INFO: `/v3/users/89583f5f42ab49058e780d3e5c44662c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:21.167571 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a787d37-a04d-4c98-b500-a257be81800c None admin] RBAC: Authorizing `identity:get_user(user_id=89583f5f42ab49058e780d3e5c44662c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:21.169053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a787d37-a04d-4c98-b500-a257be81800c None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:21.175017 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 80/157] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:20 2022] GET /identity/v3/users/89583f5f42ab49058e780d3e5c44662c => generated 275 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:21.187859 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876175c-13a5-4ffd-8b50-cde2e3db68b7 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:21.188188 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876175c-13a5-4ffd-8b50-cde2e3db68b7 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:21.188428 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876175c-13a5-4ffd-8b50-cde2e3db68b7 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:21.189938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 81/158] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:21.190674 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876175c-13a5-4ffd-8b50-cde2e3db68b7 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:21.209974 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6980b-9227-4d42-8a85-61c68affe321 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:21.246519 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6df361-054a-49bd-9844-903df2940939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:21.247529 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6df361-054a-49bd-9844-903df2940939 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:21.247811 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6df361-054a-49bd-9844-903df2940939 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:21.248085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6df361-054a-49bd-9844-903df2940939 None admin] PATH_INFO: `/v3/users/891d3014859343d8a558b44fef8a2f4f` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:21.249374 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6df361-054a-49bd-9844-903df2940939 None admin] RBAC: Authorizing `identity:get_user(user_id=891d3014859343d8a558b44fef8a2f4f)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:21.250923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6df361-054a-49bd-9844-903df2940939 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:21.252931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 78/159] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:21 2022] GET /identity/v3/users/891d3014859343d8a558b44fef8a2f4f => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:21.254344 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e479b1-326e-4cd9-bd68-05f5ce9084c5 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:21.254629 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e479b1-326e-4cd9-bd68-05f5ce9084c5 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:21.254899 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e479b1-326e-4cd9-bd68-05f5ce9084c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:21.275265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-b3e479b1-326e-4cd9-bd68-05f5ce9084c5 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:21.276475 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-b3e479b1-326e-4cd9-bd68-05f5ce9084c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:21.298831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:21.340037 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876175c-13a5-4ffd-8b50-cde2e3db68b7 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:21.378538 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 79/160] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:21.379550 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e479b1-326e-4cd9-bd68-05f5ce9084c5 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:21.385767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526727a1-cb6a-4b2c-921e-3bbc37d3bc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:21.386438 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526727a1-cb6a-4b2c-921e-3bbc37d3bc7a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:21.386722 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526727a1-cb6a-4b2c-921e-3bbc37d3bc7a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:21.386990 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526727a1-cb6a-4b2c-921e-3bbc37d3bc7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:21.401514 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:21.520758 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526727a1-cb6a-4b2c-921e-3bbc37d3bc7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:21.522300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526727a1-cb6a-4b2c-921e-3bbc37d3bc7a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:21.547683 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e479b1-326e-4cd9-bd68-05f5ce9084c5 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:21.607111 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c3e25-64d8-4f2d-8a53-ad3583b58b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:21.607838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c3e25-64d8-4f2d-8a53-ad3583b58b95 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:21.608129 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c3e25-64d8-4f2d-8a53-ad3583b58b95 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:21.608403 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c3e25-64d8-4f2d-8a53-ad3583b58b95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:21.645842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:21.705321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 82/161] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:21.705986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526727a1-cb6a-4b2c-921e-3bbc37d3bc7a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:21.717778 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c3e25-64d8-4f2d-8a53-ad3583b58b95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:21.719266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c3e25-64d8-4f2d-8a53-ad3583b58b95 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:21.767396 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:21.873236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 80/162] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:21.881280 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526727a1-cb6a-4b2c-921e-3bbc37d3bc7a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:21.882010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c3e25-64d8-4f2d-8a53-ad3583b58b95 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:21.961942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:21.982355 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1c582-1b79-4748-9218-655a29b2f348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:21.983170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1c582-1b79-4748-9218-655a29b2f348 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:21.983454 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1c582-1b79-4748-9218-655a29b2f348 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:21.983726 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1c582-1b79-4748-9218-655a29b2f348 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:22.033441 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c3e25-64d8-4f2d-8a53-ad3583b58b95 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:22.035776 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1c582-1b79-4748-9218-655a29b2f348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:22.043598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-c7d1c582-1b79-4748-9218-655a29b2f348 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:22.043598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:22.043598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:22.043598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:22.043598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:22.043598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:22.043598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:22.043598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:22.043598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:22.043598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:22.043598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:22.043598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:22.043598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:22.043598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:22.043598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:22.043598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:22.045062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:22.045062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:22.045062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:22.045062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:22.045062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:22.045062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:22.045062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:22.045062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:22.045062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:22.045062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:22.045062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:22.045062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:22.045062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:22.045062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:22.045062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:22.045062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:22.045882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:22.045882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:22.045882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:22.045882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:22.045882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:22.045882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:22.045882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:22.045882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:22.045882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:22.045882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:22.045882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:22.045882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:22.053500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 83/163] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:22.054346 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1c582-1b79-4748-9218-655a29b2f348 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:22.086492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36aeed-cc4b-4dbf-81d1-aa9164b1917e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:22.087380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36aeed-cc4b-4dbf-81d1-aa9164b1917e None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:22.087668 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36aeed-cc4b-4dbf-81d1-aa9164b1917e None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:22.087946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36aeed-cc4b-4dbf-81d1-aa9164b1917e None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:22.089231 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36aeed-cc4b-4dbf-81d1-aa9164b1917e None admin] RBAC: Authorizing `identity:get_project(project_id=3790dd61247c450ebbf3ef2e90a818e0)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:22.090713 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36aeed-cc4b-4dbf-81d1-aa9164b1917e None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:22.093033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 81/164] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:21 2022] GET /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:22.093840 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36aeed-cc4b-4dbf-81d1-aa9164b1917e None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:22.215056 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36aeed-cc4b-4dbf-81d1-aa9164b1917e None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:22.245463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:22.254806 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5961336-6b3b-4a33-b804-4376172beb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:22.255498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5961336-6b3b-4a33-b804-4376172beb7a None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:22.255838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5961336-6b3b-4a33-b804-4376172beb7a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:22.256125 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5961336-6b3b-4a33-b804-4376172beb7a None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:22.256901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5961336-6b3b-4a33-b804-4376172beb7a None admin] RBAC: Adding query filter params (name=member) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:22.257241 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5961336-6b3b-4a33-b804-4376172beb7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:22.259476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5961336-6b3b-4a33-b804-4376172beb7a None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:22.269223 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 82/165] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:22.270081 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5961336-6b3b-4a33-b804-4376172beb7a None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:22.334874 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:22.371069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1c582-1b79-4748-9218-655a29b2f348 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:22.385103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5961336-6b3b-4a33-b804-4376172beb7a None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:22.422747 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7fd390-432b-4ea9-9d30-c32065b507c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:22.423415 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fd390-432b-4ea9-9d30-c32065b507c4 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:22.423703 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fd390-432b-4ea9-9d30-c32065b507c4 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:22.423982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fd390-432b-4ea9-9d30-c32065b507c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:22.424831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7fd390-432b-4ea9-9d30-c32065b507c4 None admin] RBAC: Adding query filter params (role.id=8a0eeb2a21dc4897a58fb5851d58678c, scope.project.id=3790dd61247c450ebbf3ef2e90a818e0, user.id=89583f5f42ab49058e780d3e5c44662c) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:22.425171 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7fd390-432b-4ea9-9d30-c32065b507c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:22.427152 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7fd390-432b-4ea9-9d30-c32065b507c4 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:22.433910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 83/166] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:25:22 2022] GET /identity/v3/role_assignments?user.id=89583f5f42ab49058e780d3e5c44662c&scope.project.id=3790dd61247c450ebbf3ef2e90a818e0&role.id=8a0eeb2a21dc4897a58fb5851d58678c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:22.435403 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff071fe-926b-4100-b5b8-247ccbe3b3bb None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:22.435870 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff071fe-926b-4100-b5b8-247ccbe3b3bb None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:22.436162 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff071fe-926b-4100-b5b8-247ccbe3b3bb None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:22.442232 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 84/167] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:22.442855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff071fe-926b-4100-b5b8-247ccbe3b3bb None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:22.480299 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c1a4a-6895-4217-836d-7796d62523ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:22.481117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c1a4a-6895-4217-836d-7796d62523ce None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:22.481404 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c1a4a-6895-4217-836d-7796d62523ce None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:22.481757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c1a4a-6895-4217-836d-7796d62523ce None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:22.483010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c1a4a-6895-4217-836d-7796d62523ce None admin] RBAC: Authorizing `identity:get_project(project_id=3790dd61247c450ebbf3ef2e90a818e0)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:22.484462 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c1a4a-6895-4217-836d-7796d62523ce None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:22.492818 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 84/168] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:22 2022] GET /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0 => generated 298 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:22.494178 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff3db0-f55d-41ce-abcf-3340e19368f3 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:22.494464 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff3db0-f55d-41ce-abcf-3340e19368f3 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:22.494741 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff3db0-f55d-41ce-abcf-3340e19368f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:22.538136 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-7aff3db0-f55d-41ce-abcf-3340e19368f3 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:22.539370 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-7aff3db0-f55d-41ce-abcf-3340e19368f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:22.557537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:22.646768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff071fe-926b-4100-b5b8-247ccbe3b3bb None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:22.701832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a282ff-47dc-4e12-be38-831634295466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:22.703074 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a282ff-47dc-4e12-be38-831634295466 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:22.703578 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a282ff-47dc-4e12-be38-831634295466 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:22.704045 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a282ff-47dc-4e12-be38-831634295466 None admin] PATH_INFO: `/v3/users/89583f5f42ab49058e780d3e5c44662c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:22.709959 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a282ff-47dc-4e12-be38-831634295466 None admin] RBAC: Authorizing `identity:get_user(user_id=89583f5f42ab49058e780d3e5c44662c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:22.711379 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 85/169] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:22.712047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff3db0-f55d-41ce-abcf-3340e19368f3 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:22.712829 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a282ff-47dc-4e12-be38-831634295466 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:22.715180 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 85/170] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:22 2022] GET /identity/v3/users/89583f5f42ab49058e780d3e5c44662c => generated 275 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:22.726127 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a282ff-47dc-4e12-be38-831634295466 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:22.857469 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:22.871781 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff3db0-f55d-41ce-abcf-3340e19368f3 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:22.944919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce3459-04fc-4c7a-aab5-ab1720ab6f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:22.945611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce3459-04fc-4c7a-aab5-ab1720ab6f20 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:22.945893 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce3459-04fc-4c7a-aab5-ab1720ab6f20 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:22.946166 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce3459-04fc-4c7a-aab5-ab1720ab6f20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:22.947013 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce3459-04fc-4c7a-aab5-ab1720ab6f20 None admin] RBAC: Adding query filter params (role.id=19f5e72c0be44a1197c7288de82e79e8, scope.project.id=3790dd61247c450ebbf3ef2e90a818e0, user.id=891d3014859343d8a558b44fef8a2f4f) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:22.947325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce3459-04fc-4c7a-aab5-ab1720ab6f20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:22.948783 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce3459-04fc-4c7a-aab5-ab1720ab6f20 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:22.965478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 86/171] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:25:22 2022] GET /identity/v3/role_assignments?user.id=891d3014859343d8a558b44fef8a2f4f&scope.project.id=3790dd61247c450ebbf3ef2e90a818e0&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:22.965964 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a282ff-47dc-4e12-be38-831634295466 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:22.966574 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce3459-04fc-4c7a-aab5-ab1720ab6f20 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:23.001598 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:23.061916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7177e01-5667-4bcc-acee-12a22c2bbfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:23.062595 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7177e01-5667-4bcc-acee-12a22c2bbfdb None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:23.062876 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7177e01-5667-4bcc-acee-12a22c2bbfdb None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:23.063154 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7177e01-5667-4bcc-acee-12a22c2bbfdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:23.110740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce3459-04fc-4c7a-aab5-ab1720ab6f20 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:23.155814 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71cb41-91b4-43d3-a711-e870af68e7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:23.156647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71cb41-91b4-43d3-a711-e870af68e7e1 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:23.156927 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71cb41-91b4-43d3-a711-e870af68e7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:23.157222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71cb41-91b4-43d3-a711-e870af68e7e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:23.180842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71cb41-91b4-43d3-a711-e870af68e7e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:23.194125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-ff71cb41-91b4-43d3-a711-e870af68e7e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:23.194125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:23.194125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:23.194125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:23.194125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:23.194125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:23.194125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:23.194125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:23.194125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:23.194125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:23.194125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:23.194125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:23.194125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:23.194125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:23.194125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:23.194125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:23.195138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:23.195138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:23.195138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:23.195138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:23.195138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:23.195138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:23.195138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:23.195138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:23.195138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:23.195138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:23.195138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:23.195138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:23.195138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:23.195138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:23.195138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:23.195138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:23.195936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:23.195936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:23.195936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:23.195936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:23.195936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:23.195936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:23.195936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:23.195936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:23.195936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:23.195936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:23.195936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:23.195936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:23.204695 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 87/172] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:25:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:23.206109 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd520f27-260e-4545-a657-eae017dd1f13 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:23.206366 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd520f27-260e-4545-a657-eae017dd1f13 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:23.206643 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd520f27-260e-4545-a657-eae017dd1f13 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:23.207930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 88/173] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:23.215072 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd520f27-260e-4545-a657-eae017dd1f13 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:23.216295 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7177e01-5667-4bcc-acee-12a22c2bbfdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:23.217820 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7177e01-5667-4bcc-acee-12a22c2bbfdb None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:23.257500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:23.424124 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 86/174] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 698 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:23.426228 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4cb7a-e6ae-4dc4-878f-b710277e0cc4 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:23.426522 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4cb7a-e6ae-4dc4-878f-b710277e0cc4 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:23.426799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4cb7a-e6ae-4dc4-878f-b710277e0cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:23.428639 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd520f27-260e-4545-a657-eae017dd1f13 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:23.466769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-94f4cb7a-e6ae-4dc4-878f-b710277e0cc4 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:23.468097 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-94f4cb7a-e6ae-4dc4-878f-b710277e0cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:23.493480 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:23.508528 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340209f-a2aa-4541-81d4-d6fb07471650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:23.509247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340209f-a2aa-4541-81d4-d6fb07471650 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:23.509534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340209f-a2aa-4541-81d4-d6fb07471650 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:23.509812 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340209f-a2aa-4541-81d4-d6fb07471650 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:23.510645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340209f-a2aa-4541-81d4-d6fb07471650 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:23.510954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340209f-a2aa-4541-81d4-d6fb07471650 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:23.514048 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340209f-a2aa-4541-81d4-d6fb07471650 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:23.533863 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 89/175] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:25:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:23.534805 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340209f-a2aa-4541-81d4-d6fb07471650 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:23.606161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 87/176] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:23.608200 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4cb7a-e6ae-4dc4-878f-b710277e0cc4 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:23.705541 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:23.735912 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340209f-a2aa-4541-81d4-d6fb07471650 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:23.775125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b90c51-7b36-4130-8022-af74a6e1961d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:23.776035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b90c51-7b36-4130-8022-af74a6e1961d None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:23.776389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b90c51-7b36-4130-8022-af74a6e1961d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:23.776677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b90c51-7b36-4130-8022-af74a6e1961d None admin] PATH_INFO: `/v3/projects/7556db6b05094000a819215f2f3e7f47` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:23.777930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b90c51-7b36-4130-8022-af74a6e1961d None admin] RBAC: Authorizing `identity:get_project(project_id=7556db6b05094000a819215f2f3e7f47)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:23.780174 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b90c51-7b36-4130-8022-af74a6e1961d None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:23.782253 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 90/177] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:23 2022] GET /identity/v3/projects/7556db6b05094000a819215f2f3e7f47 => generated 312 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:23.782909 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b90c51-7b36-4130-8022-af74a6e1961d None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:23.790136 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4cb7a-e6ae-4dc4-878f-b710277e0cc4 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:23.829154 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0a6e8-492d-4e42-af47-4fb0f6a319a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:23.830041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0a6e8-492d-4e42-af47-4fb0f6a319a4 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:23.830322 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0a6e8-492d-4e42-af47-4fb0f6a319a4 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:23.830596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0a6e8-492d-4e42-af47-4fb0f6a319a4 None admin] PATH_INFO: `/v3/users/89583f5f42ab49058e780d3e5c44662c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:23.831896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0a6e8-492d-4e42-af47-4fb0f6a319a4 None admin] RBAC: Authorizing `identity:get_user(user_id=89583f5f42ab49058e780d3e5c44662c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:23.833517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0a6e8-492d-4e42-af47-4fb0f6a319a4 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:23.835226 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 88/178] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:23 2022] GET /identity/v3/users/89583f5f42ab49058e780d3e5c44662c => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:23.836478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e37418-0e8c-4162-a2d4-5dd5e1b3b2ff None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:23.836786 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e37418-0e8c-4162-a2d4-5dd5e1b3b2ff None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:23.837111 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e37418-0e8c-4162-a2d4-5dd5e1b3b2ff None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:23.838676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 89/179] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:23.843297 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e37418-0e8c-4162-a2d4-5dd5e1b3b2ff None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:23.862504 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:23.907799 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b90c51-7b36-4130-8022-af74a6e1961d None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:23.946453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0e222-2550-4e61-b9e3-96b16f534c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:23.947745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0e222-2550-4e61-b9e3-96b16f534c4a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:23.948250 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0e222-2550-4e61-b9e3-96b16f534c4a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:23.948783 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0e222-2550-4e61-b9e3-96b16f534c4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:23.956627 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e37418-0e8c-4162-a2d4-5dd5e1b3b2ff None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:23.963052 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0e222-2550-4e61-b9e3-96b16f534c4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:23.972704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-8fd0e222-2550-4e61-b9e3-96b16f534c4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:23.972704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:23.972704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:23.972704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:23.972704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:23.972704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:23.972704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:23.972704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:23.972704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:23.972704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:23.972704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:23.972704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:23.972704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:23.972704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:23.972704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:23.972704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:23.973662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:23.973662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:23.973662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:23.973662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:23.973662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:23.973662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:23.973662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:23.973662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:23.973662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:23.973662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:23.973662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:23.973662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:23.973662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:23.973662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:23.973662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:23.973662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:23.974727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:23.974727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:23.974727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:23.974727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:23.974727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:23.974727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:23.974727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:23.974727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:23.974727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:23.974727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:23.974727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:23.974727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:23.977283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 91/180] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:25:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:23.978285 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0e222-2550-4e61-b9e3-96b16f534c4a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:23.990702 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:23.994651 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417d756-d88f-4c62-b07a-e862d171a589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:23.995459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417d756-d88f-4c62-b07a-e862d171a589 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:23.995821 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417d756-d88f-4c62-b07a-e862d171a589 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:23.996172 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417d756-d88f-4c62-b07a-e862d171a589 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:23.997135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417d756-d88f-4c62-b07a-e862d171a589 None admin] RBAC: Adding query filter params (role.id=8a0eeb2a21dc4897a58fb5851d58678c, scope.project.id=7556db6b05094000a819215f2f3e7f47, user.id=89583f5f42ab49058e780d3e5c44662c) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:23.998032 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417d756-d88f-4c62-b07a-e862d171a589 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:23.999575 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417d756-d88f-4c62-b07a-e862d171a589 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:24.013570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 90/181] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:25:23 2022] GET /identity/v3/role_assignments?user.id=89583f5f42ab49058e780d3e5c44662c&scope.project.id=7556db6b05094000a819215f2f3e7f47&role.id=8a0eeb2a21dc4897a58fb5851d58678c => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:24.013570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18e61c-55d6-43da-aec7-67543cd3f70a None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:24.013570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18e61c-55d6-43da-aec7-67543cd3f70a None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:24.013570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18e61c-55d6-43da-aec7-67543cd3f70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:24.042190 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-5d18e61c-55d6-43da-aec7-67543cd3f70a None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:24.045165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-5d18e61c-55d6-43da-aec7-67543cd3f70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:24.129535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0e222-2550-4e61-b9e3-96b16f534c4a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:24.146135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 91/182] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:24.147427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18e61c-55d6-43da-aec7-67543cd3f70a None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:24.158751 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:24.168764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642957c-64b0-42cf-a9e2-ac194bf7c299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:24.169998 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642957c-64b0-42cf-a9e2-ac194bf7c299 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:24.170432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642957c-64b0-42cf-a9e2-ac194bf7c299 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:24.170601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642957c-64b0-42cf-a9e2-ac194bf7c299 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:24.256512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642957c-64b0-42cf-a9e2-ac194bf7c299 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:24.257964 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642957c-64b0-42cf-a9e2-ac194bf7c299 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:24.266427 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:24.284420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18e61c-55d6-43da-aec7-67543cd3f70a None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:24.321915 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b77aa-3ff8-4285-a998-173711cbb2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:24.322596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b77aa-3ff8-4285-a998-173711cbb2df None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:24.322881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b77aa-3ff8-4285-a998-173711cbb2df None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:24.323156 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b77aa-3ff8-4285-a998-173711cbb2df None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:24.323916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b77aa-3ff8-4285-a998-173711cbb2df None admin] RBAC: Adding query filter params (name=admin) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:24.324223 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b77aa-3ff8-4285-a998-173711cbb2df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:24.325670 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b77aa-3ff8-4285-a998-173711cbb2df None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:24.334065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 92/183] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:24.339929 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa47485-4ace-46d8-bd5c-588c869299a6 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:24.340225 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa47485-4ace-46d8-bd5c-588c869299a6 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:24.340509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa47485-4ace-46d8-bd5c-588c869299a6 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:24.341873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 93/184] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:24.342576 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa47485-4ace-46d8-bd5c-588c869299a6 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:24.364980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 92/185] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:24.366336 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae07c1-24fd-4c04-89fa-2834fd200e66 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:24.366622 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae07c1-24fd-4c04-89fa-2834fd200e66 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:24.366904 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae07c1-24fd-4c04-89fa-2834fd200e66 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:24.368179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 93/186] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:24.368872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae07c1-24fd-4c04-89fa-2834fd200e66 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:24.412814 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:24.450519 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa47485-4ace-46d8-bd5c-588c869299a6 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:24.470327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae07c1-24fd-4c04-89fa-2834fd200e66 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:24.481256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:24.487103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a596c-2ab6-4490-a242-0df560402a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:24.487907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a596c-2ab6-4490-a242-0df560402a10 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:24.488189 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a596c-2ab6-4490-a242-0df560402a10 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:24.488469 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a596c-2ab6-4490-a242-0df560402a10 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:24.498861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a596c-2ab6-4490-a242-0df560402a10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:24.506785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-251a596c-2ab6-4490-a242-0df560402a10 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:25:24.506785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:24.506785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:24.506785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:24.506785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:24.506785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:24.506785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:24.506785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:24.506785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:24.506785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:24.506785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:24.506785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:24.506785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:24.506785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:24.506785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:24.506785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:24.507946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:24.507946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:24.507946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:24.507946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:24.507946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:24.507946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:24.507946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:24.507946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:24.507946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:24.507946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:24.507946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:24.507946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:24.507946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:24.507946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:24.507946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:24.507946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:24.508921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:24.508921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:24.508921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:24.508921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:24.508921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:24.508921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:24.508921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:24.508921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:24.508921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:24.508921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:24.508921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:25:24.508921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:24.508921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 94/187] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:24.509743 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80079f26-94d4-4d26-8cc2-8fce49c620ca None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:24.509743 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80079f26-94d4-4d26-8cc2-8fce49c620ca None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:24.509743 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80079f26-94d4-4d26-8cc2-8fce49c620ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:24.511990 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d426353-d0d3-4341-b4a8-176bbde1a6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:24.513107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d426353-d0d3-4341-b4a8-176bbde1a6a1 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:24.513393 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d426353-d0d3-4341-b4a8-176bbde1a6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:24.513668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d426353-d0d3-4341-b4a8-176bbde1a6a1 None admin] PATH_INFO: `/v3/users/81e7e312cce349e8b47efbbeb888c186` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:24.525846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-80079f26-94d4-4d26-8cc2-8fce49c620ca None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:24.527042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-80079f26-94d4-4d26-8cc2-8fce49c620ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:24.529203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d426353-d0d3-4341-b4a8-176bbde1a6a1 None admin] RBAC: Authorizing `identity:get_user(user_id=81e7e312cce349e8b47efbbeb888c186)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:24.530656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d426353-d0d3-4341-b4a8-176bbde1a6a1 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:24.532327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 94/188] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:24 2022] GET /identity/v3/users/81e7e312cce349e8b47efbbeb888c186 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:24.533108 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d426353-d0d3-4341-b4a8-176bbde1a6a1 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:24.596371 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 95/189] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:24.597779 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9f6d5-5454-4692-85bf-27eb0eea2768 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:24.598146 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9f6d5-5454-4692-85bf-27eb0eea2768 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:24.598495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9f6d5-5454-4692-85bf-27eb0eea2768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:24.614674 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-28c9f6d5-5454-4692-85bf-27eb0eea2768 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:24.615958 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-28c9f6d5-5454-4692-85bf-27eb0eea2768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:24.632963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d426353-d0d3-4341-b4a8-176bbde1a6a1 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:24.668832 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea1aa1-f134-4f74-9e6f-18b72fee7c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:24.669870 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea1aa1-f134-4f74-9e6f-18b72fee7c64 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:24.670292 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea1aa1-f134-4f74-9e6f-18b72fee7c64 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:24.670421 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:24.670775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea1aa1-f134-4f74-9e6f-18b72fee7c64 None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:24.672620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea1aa1-f134-4f74-9e6f-18b72fee7c64 None admin] RBAC: Authorizing `identity:get_project(project_id=3790dd61247c450ebbf3ef2e90a818e0)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:24.674402 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea1aa1-f134-4f74-9e6f-18b72fee7c64 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:24.676967 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 95/190] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:24 2022] GET /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:24.678126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea1aa1-f134-4f74-9e6f-18b72fee7c64 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:24.689714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 96/191] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:24.698734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9f6d5-5454-4692-85bf-27eb0eea2768 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:24.722818 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:24.791760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9f6d5-5454-4692-85bf-27eb0eea2768 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:24.795140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea1aa1-f134-4f74-9e6f-18b72fee7c64 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:24.826523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4154325-471c-470c-96d6-48338e54e961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:24.827198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4154325-471c-470c-96d6-48338e54e961 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:24.827485 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4154325-471c-470c-96d6-48338e54e961 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:24.827754 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4154325-471c-470c-96d6-48338e54e961 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:24.829909 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e48ce7-4a97-4f01-807a-eb299e1f29bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:24.830574 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e48ce7-4a97-4f01-807a-eb299e1f29bb None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:24.830855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e48ce7-4a97-4f01-807a-eb299e1f29bb None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:24.831130 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e48ce7-4a97-4f01-807a-eb299e1f29bb None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:24.831873 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e48ce7-4a97-4f01-807a-eb299e1f29bb None admin] RBAC: Adding query filter params (name=reader) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:24.832182 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e48ce7-4a97-4f01-807a-eb299e1f29bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:24.833588 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e48ce7-4a97-4f01-807a-eb299e1f29bb None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:24.842606 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 96/192] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:24.843394 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e48ce7-4a97-4f01-807a-eb299e1f29bb None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:24.882448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:24.896074 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4154325-471c-470c-96d6-48338e54e961 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:24.897609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4154325-471c-470c-96d6-48338e54e961 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:24.937728 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:24.945671 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e48ce7-4a97-4f01-807a-eb299e1f29bb None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:24.980572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcdfcf0-8d3a-4ea8-81f5-2fa5f7e8b1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:24.981395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdfcf0-8d3a-4ea8-81f5-2fa5f7e8b1f9 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:24.981699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdfcf0-8d3a-4ea8-81f5-2fa5f7e8b1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:24.981969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdfcf0-8d3a-4ea8-81f5-2fa5f7e8b1f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:24.983126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcdfcf0-8d3a-4ea8-81f5-2fa5f7e8b1f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:24.984566 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcdfcf0-8d3a-4ea8-81f5-2fa5f7e8b1f9 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:24.986192 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 97/193] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:25:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:24.986946 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcdfcf0-8d3a-4ea8-81f5-2fa5f7e8b1f9 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:25.001893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 97/194] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:25.002744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4154325-471c-470c-96d6-48338e54e961 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:25.085337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcdfcf0-8d3a-4ea8-81f5-2fa5f7e8b1f9 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:25.091871 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:25.104454 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4154325-471c-470c-96d6-48338e54e961 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:25.119298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1b5ab-494d-433b-9030-0d941fdee78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:25.119944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1b5ab-494d-433b-9030-0d941fdee78e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:25.120227 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1b5ab-494d-433b-9030-0d941fdee78e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:25.120507 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1b5ab-494d-433b-9030-0d941fdee78e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:25.121382 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1b5ab-494d-433b-9030-0d941fdee78e None admin] RBAC: Adding query filter params (role.id=bad7164dd96e4048bab9360ef3f6769e, scope.project.id=3790dd61247c450ebbf3ef2e90a818e0, user.id=89583f5f42ab49058e780d3e5c44662c) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:25.121696 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1b5ab-494d-433b-9030-0d941fdee78e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:25.123166 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1b5ab-494d-433b-9030-0d941fdee78e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:25.129102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 98/195] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:25:24 2022] GET /identity/v3/role_assignments?user.id=89583f5f42ab49058e780d3e5c44662c&scope.project.id=3790dd61247c450ebbf3ef2e90a818e0&role.id=bad7164dd96e4048bab9360ef3f6769e => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:25.129730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1b5ab-494d-433b-9030-0d941fdee78e None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:25.150897 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124415b7-780a-4604-ac49-6114dbb6abc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:25.151734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124415b7-780a-4604-ac49-6114dbb6abc4 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:25.152023 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124415b7-780a-4604-ac49-6114dbb6abc4 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:25.152302 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124415b7-780a-4604-ac49-6114dbb6abc4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:25.161973 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:25.166613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124415b7-780a-4604-ac49-6114dbb6abc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:25.175029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-124415b7-780a-4604-ac49-6114dbb6abc4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:25.175029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:25.175029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:25.175029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:25.175029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:25.175029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:25.175029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:25.175029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:25.175029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:25.175029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:25.175029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:25.175029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:25.175029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:25.175029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:25.175029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:25.175029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:25.176010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:25.176010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:25.176010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:25.176010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:25.176010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:25.176010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:25.176010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:25.176010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:25.176010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:25.176010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:25.176010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:25.176010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:25.176010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:25.176010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:25.176010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:25.176010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:25.176832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:25.176832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:25.176832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:25.176832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:25.176832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:25.176832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:25.176832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:25.176832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:25.176832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:25.176832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:25.176832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:25.176832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:25.176832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 98/196] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:25:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:25.177858 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124415b7-780a-4604-ac49-6114dbb6abc4 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:25.252334 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1b5ab-494d-433b-9030-0d941fdee78e None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:25.292024 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ca4ad-12e2-4f0a-b259-6bccb9ca32ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:25.293102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ca4ad-12e2-4f0a-b259-6bccb9ca32ed None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:25.293232 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ca4ad-12e2-4f0a-b259-6bccb9ca32ed None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:25.293473 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ca4ad-12e2-4f0a-b259-6bccb9ca32ed None admin] PATH_INFO: `/v3/users/2042d6d62f0e47fdb382f8a7ab759b9c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:25.294878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ca4ad-12e2-4f0a-b259-6bccb9ca32ed None admin] RBAC: Authorizing `identity:get_user(user_id=2042d6d62f0e47fdb382f8a7ab759b9c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:25.296569 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ca4ad-12e2-4f0a-b259-6bccb9ca32ed None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:25.299098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 99/197] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:25 2022] GET /identity/v3/users/2042d6d62f0e47fdb382f8a7ab759b9c => generated 289 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:25.299809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ca4ad-12e2-4f0a-b259-6bccb9ca32ed None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:25.302221 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124415b7-780a-4604-ac49-6114dbb6abc4 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:25.346927 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095213b4-d5b7-4d1b-ae64-2ab757bcb042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:25.347823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095213b4-d5b7-4d1b-ae64-2ab757bcb042 None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:25.348281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095213b4-d5b7-4d1b-ae64-2ab757bcb042 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:25.348560 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095213b4-d5b7-4d1b-ae64-2ab757bcb042 None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:25.349583 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095213b4-d5b7-4d1b-ae64-2ab757bcb042 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:25.351038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095213b4-d5b7-4d1b-ae64-2ab757bcb042 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:25.379193 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 99/198] 10.222.0.71 () {60 vars in 1132 bytes} [Thu Jul 21 20:25:25 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:25:25.380329 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095213b4-d5b7-4d1b-ae64-2ab757bcb042 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:25.393687 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:25.410572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:25.421537 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ca4ad-12e2-4f0a-b259-6bccb9ca32ed None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:25.468915 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68570c2-dd75-49e5-910a-3a6fe185afff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:25.470792 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68570c2-dd75-49e5-910a-3a6fe185afff None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:25.470792 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68570c2-dd75-49e5-910a-3a6fe185afff None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:25.470792 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68570c2-dd75-49e5-910a-3a6fe185afff None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:25.472411 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68570c2-dd75-49e5-910a-3a6fe185afff None admin] RBAC: Authorizing `identity:get_project(project_id=956f270e083e4b15b70833346dcb8e19)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:25.477261 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68570c2-dd75-49e5-910a-3a6fe185afff None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:25.479299 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 100/199] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:25 2022] GET /identity/v3/projects/956f270e083e4b15b70833346dcb8e19 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:25.484954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68570c2-dd75-49e5-910a-3a6fe185afff None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:25.500376 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095213b4-d5b7-4d1b-ae64-2ab757bcb042 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:25.541116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a470a8-ec01-4458-a9a6-60b8f1f554e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:25.541527 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a470a8-ec01-4458-a9a6-60b8f1f554e0 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:25.541814 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a470a8-ec01-4458-a9a6-60b8f1f554e0 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:25.542405 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a470a8-ec01-4458-a9a6-60b8f1f554e0 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:25.542856 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a470a8-ec01-4458-a9a6-60b8f1f554e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:25.543169 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a470a8-ec01-4458-a9a6-60b8f1f554e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:25.544647 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a470a8-ec01-4458-a9a6-60b8f1f554e0 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:25.555431 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 100/200] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:25:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:25.556275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a470a8-ec01-4458-a9a6-60b8f1f554e0 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:25.602748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68570c2-dd75-49e5-910a-3a6fe185afff None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:25.639227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:25.644864 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0766080-9e0f-4fa5-aa2f-b64a1d5695d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:25.645549 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0766080-9e0f-4fa5-aa2f-b64a1d5695d8 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:25.645831 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0766080-9e0f-4fa5-aa2f-b64a1d5695d8 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:25.646106 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0766080-9e0f-4fa5-aa2f-b64a1d5695d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:25.678655 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a470a8-ec01-4458-a9a6-60b8f1f554e0 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:25.680497 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:25.723475 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0766080-9e0f-4fa5-aa2f-b64a1d5695d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:25.724619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec0f26-5b4e-4335-8fd3-9ac3720a7b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:25.725261 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0766080-9e0f-4fa5-aa2f-b64a1d5695d8 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:25.725450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec0f26-5b4e-4335-8fd3-9ac3720a7b19 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:25.725685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec0f26-5b4e-4335-8fd3-9ac3720a7b19 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:25.725967 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec0f26-5b4e-4335-8fd3-9ac3720a7b19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:25.726819 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec0f26-5b4e-4335-8fd3-9ac3720a7b19 None admin] RBAC: Adding query filter params (role.id=19f5e72c0be44a1197c7288de82e79e8, scope.project.id=956f270e083e4b15b70833346dcb8e19, user.id=81e7e312cce349e8b47efbbeb888c186) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:25.727137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec0f26-5b4e-4335-8fd3-9ac3720a7b19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:25.728599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec0f26-5b4e-4335-8fd3-9ac3720a7b19 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:25.737485 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 101/201] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:25:25 2022] GET /identity/v3/role_assignments?user.id=81e7e312cce349e8b47efbbeb888c186&scope.project.id=956f270e083e4b15b70833346dcb8e19&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:25.738330 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec0f26-5b4e-4335-8fd3-9ac3720a7b19 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:25.866190 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 101/202] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:25.867913 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec0f26-5b4e-4335-8fd3-9ac3720a7b19 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:25.874869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0766080-9e0f-4fa5-aa2f-b64a1d5695d8 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:25.905024 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:25.923290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccd421-9edc-4b5b-8288-11c72ff4f87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:25.924185 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccd421-9edc-4b5b-8288-11c72ff4f87b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:25.924471 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccd421-9edc-4b5b-8288-11c72ff4f87b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:25.924751 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccd421-9edc-4b5b-8288-11c72ff4f87b None admin] PATH_INFO: `/v3/users/81e7e312cce349e8b47efbbeb888c186` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:25.929956 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eccd421-9edc-4b5b-8288-11c72ff4f87b None admin] RBAC: Authorizing `identity:get_user(user_id=81e7e312cce349e8b47efbbeb888c186)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:25.931461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eccd421-9edc-4b5b-8288-11c72ff4f87b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:25.933405 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 102/203] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:25 2022] GET /identity/v3/users/81e7e312cce349e8b47efbbeb888c186 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:25.939357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccd421-9edc-4b5b-8288-11c72ff4f87b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:25.963958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:26.007530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0766080-9e0f-4fa5-aa2f-b64a1d5695d8 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:26.050006 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d180b60-5441-495b-9315-6c5eeb42bae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:26.050881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d180b60-5441-495b-9315-6c5eeb42bae5 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:26.051158 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d180b60-5441-495b-9315-6c5eeb42bae5 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:26.051527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d180b60-5441-495b-9315-6c5eeb42bae5 None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:26.057910 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d180b60-5441-495b-9315-6c5eeb42bae5 None admin] RBAC: Authorizing `identity:get_project(project_id=3790dd61247c450ebbf3ef2e90a818e0)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:26.059500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d180b60-5441-495b-9315-6c5eeb42bae5 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:26.062240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 102/204] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:25 2022] GET /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:26.069367 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d180b60-5441-495b-9315-6c5eeb42bae5 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:26.078230 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccd421-9edc-4b5b-8288-11c72ff4f87b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:26.123087 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910029c-c610-4f67-a181-ea40f48a0d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:26.123669 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910029c-c610-4f67-a181-ea40f48a0d24 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:26.123951 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910029c-c610-4f67-a181-ea40f48a0d24 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:26.124227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910029c-c610-4f67-a181-ea40f48a0d24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:26.212317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d180b60-5441-495b-9315-6c5eeb42bae5 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:26.246118 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:26.254270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac65a7-d721-45aa-9a33-c986160deba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:26.254998 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac65a7-d721-45aa-9a33-c986160deba7 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:26.255287 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac65a7-d721-45aa-9a33-c986160deba7 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:26.255562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac65a7-d721-45aa-9a33-c986160deba7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:26.256412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac65a7-d721-45aa-9a33-c986160deba7 None admin] RBAC: Adding query filter params (role.id=bc82ed55215d47b791d5aa25699e7e2b, scope.project.id=3790dd61247c450ebbf3ef2e90a818e0, user.id=2042d6d62f0e47fdb382f8a7ab759b9c) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:26.256726 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac65a7-d721-45aa-9a33-c986160deba7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:26.258514 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:26.258719 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac65a7-d721-45aa-9a33-c986160deba7 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:26.263760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910029c-c610-4f67-a181-ea40f48a0d24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:26.265554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910029c-c610-4f67-a181-ea40f48a0d24 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:26.267105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 103/205] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:25:26 2022] GET /identity/v3/role_assignments?user.id=2042d6d62f0e47fdb382f8a7ab759b9c&scope.project.id=3790dd61247c450ebbf3ef2e90a818e0&role.id=bc82ed55215d47b791d5aa25699e7e2b => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:26.430733 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 103/206] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 20:25:26.436824 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac65a7-d721-45aa-9a33-c986160deba7 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:26.565932 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac65a7-d721-45aa-9a33-c986160deba7 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:26.602259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46787dcf-8b8a-45ec-af01-7a44a3018d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:26.603560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46787dcf-8b8a-45ec-af01-7a44a3018d21 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:26.604054 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46787dcf-8b8a-45ec-af01-7a44a3018d21 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:26.604527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46787dcf-8b8a-45ec-af01-7a44a3018d21 None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:26.606015 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46787dcf-8b8a-45ec-af01-7a44a3018d21 None admin] RBAC: Authorizing `identity:get_project(project_id=956f270e083e4b15b70833346dcb8e19)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:26.608213 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46787dcf-8b8a-45ec-af01-7a44a3018d21 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:26.610174 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 104/207] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:26 2022] GET /identity/v3/projects/956f270e083e4b15b70833346dcb8e19 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:26.616221 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910029c-c610-4f67-a181-ea40f48a0d24 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:26.732928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:26.737805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910029c-c610-4f67-a181-ea40f48a0d24 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:26.779367 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c09ee1-6916-4156-a57f-823d6e0875bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:26.780058 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c09ee1-6916-4156-a57f-823d6e0875bd None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:26.780339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c09ee1-6916-4156-a57f-823d6e0875bd None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:26.780614 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c09ee1-6916-4156-a57f-823d6e0875bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:26.781619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c09ee1-6916-4156-a57f-823d6e0875bd None admin] RBAC: Adding query filter params (role.id=bad7164dd96e4048bab9360ef3f6769e, scope.project.id=956f270e083e4b15b70833346dcb8e19, user.id=81e7e312cce349e8b47efbbeb888c186) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:26.781934 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c09ee1-6916-4156-a57f-823d6e0875bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:26.783482 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c09ee1-6916-4156-a57f-823d6e0875bd None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:26.791579 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 104/208] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:25:26 2022] GET /identity/v3/role_assignments?user.id=81e7e312cce349e8b47efbbeb888c186&scope.project.id=956f270e083e4b15b70833346dcb8e19&role.id=bad7164dd96e4048bab9360ef3f6769e => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:27.727331 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3d4c3-0f0b-4e36-a598-39e005d28cfe None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:27.727817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3d4c3-0f0b-4e36-a598-39e005d28cfe None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:27.727817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3d4c3-0f0b-4e36-a598-39e005d28cfe None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:27.729431 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 105/209] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:27.734932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990f2a5-9af2-43bd-bbe0-87388dac2691 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:27.735451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990f2a5-9af2-43bd-bbe0-87388dac2691 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:27.735695 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990f2a5-9af2-43bd-bbe0-87388dac2691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:27.753175 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-7990f2a5-9af2-43bd-bbe0-87388dac2691 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:27.754393 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-7990f2a5-9af2-43bd-bbe0-87388dac2691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:27.824274 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 105/210] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:27.847882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3d4c3-0f0b-4e36-a598-39e005d28cfe None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:27.876609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f8e4b-0d64-4465-b925-fa6c4de2a06b None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:27.876911 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f8e4b-0d64-4465-b925-fa6c4de2a06b None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:27.877199 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f8e4b-0d64-4465-b925-fa6c4de2a06b None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:27.878591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 106/211] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:25:27.884031 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841eb4ad-fa36-4cb6-b857-646c3fa09984 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:27.884325 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841eb4ad-fa36-4cb6-b857-646c3fa09984 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:27.884603 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841eb4ad-fa36-4cb6-b857-646c3fa09984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:27.904087 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-841eb4ad-fa36-4cb6-b857-646c3fa09984 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:27.905552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-841eb4ad-fa36-4cb6-b857-646c3fa09984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:27.921206 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:27.963642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3d4c3-0f0b-4e36-a598-39e005d28cfe None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:27.982610 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 107/212] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:27.999471 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85e0f9-6c25-4756-8d03-9176a4ba72c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:28.000346 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85e0f9-6c25-4756-8d03-9176a4ba72c5 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:28.000696 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85e0f9-6c25-4756-8d03-9176a4ba72c5 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:28.001037 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85e0f9-6c25-4756-8d03-9176a4ba72c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:28.008830 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841eb4ad-fa36-4cb6-b857-646c3fa09984 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:28.012171 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85e0f9-6c25-4756-8d03-9176a4ba72c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:28.020391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-7d85e0f9-6c25-4756-8d03-9176a4ba72c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:28.020391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:28.020391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:28.020391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:28.020391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:28.020391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:28.020391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:28.020391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:28.020391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:28.020391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:28.020391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:28.020391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:28.020391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:28.020391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:28.020391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:28.020391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:28.021322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:28.021322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:28.021322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:28.021322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:28.021322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:28.021322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:28.021322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:28.021322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:28.021322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:28.021322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:28.021322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:28.021322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:28.021322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:28.021322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:28.021322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:28.021322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:28.022175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:28.022175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:28.022175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:28.022175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:28.022175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:28.022175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:28.022175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:28.022175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:28.022175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:28.022175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:28.022175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:28.022175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:28.022175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 106/213] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:28.027252 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85e0f9-6c25-4756-8d03-9176a4ba72c5 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:28.045503 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:28.118812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841eb4ad-fa36-4cb6-b857-646c3fa09984 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:28.129679 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85e0f9-6c25-4756-8d03-9176a4ba72c5 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:28.152954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:28.164797 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2ed52-fe89-480e-8cad-2f3e3633a3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:28.165666 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b001fb-e58e-42bb-aed2-354f9c8231a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:28.166029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2ed52-fe89-480e-8cad-2f3e3633a3e2 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:28.166326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b001fb-e58e-42bb-aed2-354f9c8231a2 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:28.166607 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b001fb-e58e-42bb-aed2-354f9c8231a2 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:28.166713 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2ed52-fe89-480e-8cad-2f3e3633a3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:28.166875 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b001fb-e58e-42bb-aed2-354f9c8231a2 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:28.167251 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2ed52-fe89-480e-8cad-2f3e3633a3e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:28.167624 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b001fb-e58e-42bb-aed2-354f9c8231a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:28.167928 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b001fb-e58e-42bb-aed2-354f9c8231a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:28.168867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2ed52-fe89-480e-8cad-2f3e3633a3e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:28.169491 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b001fb-e58e-42bb-aed2-354f9c8231a2 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:28.170949 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2ed52-fe89-480e-8cad-2f3e3633a3e2 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:28.173248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 108/214] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:25:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:28.181120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 107/215] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:28.181276 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2ed52-fe89-480e-8cad-2f3e3633a3e2 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:28.193121 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b001fb-e58e-42bb-aed2-354f9c8231a2 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:28.289579 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:28.300377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2ed52-fe89-480e-8cad-2f3e3633a3e2 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:28.303322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b001fb-e58e-42bb-aed2-354f9c8231a2 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:28.336937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee48a69-029f-4d30-998c-b840f6a5766b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:28.337680 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee48a69-029f-4d30-998c-b840f6a5766b None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:28.337926 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee48a69-029f-4d30-998c-b840f6a5766b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:28.338200 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee48a69-029f-4d30-998c-b840f6a5766b None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:28.338927 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee48a69-029f-4d30-998c-b840f6a5766b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:28.340376 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee48a69-029f-4d30-998c-b840f6a5766b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:28.340562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658d438-776b-4aea-891f-b3779c0aa452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:28.341513 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658d438-776b-4aea-891f-b3779c0aa452 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:28.341812 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658d438-776b-4aea-891f-b3779c0aa452 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:28.342068 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658d438-776b-4aea-891f-b3779c0aa452 None admin] PATH_INFO: `/v3/users/ef5705177006412fb518485c2913ee25` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:28.357487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658d438-776b-4aea-891f-b3779c0aa452 None admin] RBAC: Authorizing `identity:get_user(user_id=ef5705177006412fb518485c2913ee25)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:28.359411 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658d438-776b-4aea-891f-b3779c0aa452 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:28.361794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 108/216] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:28 2022] GET /identity/v3/users/ef5705177006412fb518485c2913ee25 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:28.363560 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 109/217] 10.222.0.71 () {60 vars in 1132 bytes} [Thu Jul 21 20:25:28 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:25:28.366683 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dd7b8-b5f6-4878-826e-af77e928b141 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:28.367133 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dd7b8-b5f6-4878-826e-af77e928b141 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:28.367579 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dd7b8-b5f6-4878-826e-af77e928b141 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:28.369013 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee48a69-029f-4d30-998c-b840f6a5766b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:28.369973 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 109/218] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:28.376564 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346dcdc-9cec-48ca-95e7-29d0796b3544 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:28.376902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346dcdc-9cec-48ca-95e7-29d0796b3544 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:28.377134 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346dcdc-9cec-48ca-95e7-29d0796b3544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:28.399272 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-4346dcdc-9cec-48ca-95e7-29d0796b3544 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:28.400494 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-4346dcdc-9cec-48ca-95e7-29d0796b3544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:28.449027 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:28.500344 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 110/219] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:28.523005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346dcdc-9cec-48ca-95e7-29d0796b3544 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:28.536904 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee48a69-029f-4d30-998c-b840f6a5766b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:28.592173 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bd9ab-24aa-49c8-9025-694a47d5ad5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:28.592644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bd9ab-24aa-49c8-9025-694a47d5ad5e None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:28.594328 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bd9ab-24aa-49c8-9025-694a47d5ad5e None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:28.594328 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bd9ab-24aa-49c8-9025-694a47d5ad5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:28.616266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:28.654903 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346dcdc-9cec-48ca-95e7-29d0796b3544 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:28.684408 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bd9ab-24aa-49c8-9025-694a47d5ad5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:28.686418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bd9ab-24aa-49c8-9025-694a47d5ad5e None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:28.693813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0fb67-ea34-467b-9a2f-9193a6bd6e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:28.694630 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0fb67-ea34-467b-9a2f-9193a6bd6e4d None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:28.694913 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0fb67-ea34-467b-9a2f-9193a6bd6e4d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:28.695192 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0fb67-ea34-467b-9a2f-9193a6bd6e4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:28.708189 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0fb67-ea34-467b-9a2f-9193a6bd6e4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:28.716084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-e3d0fb67-ea34-467b-9a2f-9193a6bd6e4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:28.716084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:28.716084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:28.716084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:28.716084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:28.716084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:28.716084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:28.716084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:28.716084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:28.716084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:28.716084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:28.716084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:28.716084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:28.716084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:28.716084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:28.716084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:28.717026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:28.717026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:28.717026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:28.717026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:28.717026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:28.717026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:28.717026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:28.717026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:28.717026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:28.717026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:28.717026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:28.717026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:28.717026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:28.717026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:28.717026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:28.717026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:28.717968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:28.717968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:28.717968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:28.717968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:28.717968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:28.717968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:28.717968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:28.717968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:28.717968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:28.717968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:28.717968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:28.717968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:28.718595 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 111/220] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:25:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:25:28.724612 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0fb67-ea34-467b-9a2f-9193a6bd6e4d None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:28.765756 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:28.814131 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 110/221] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:28.822969 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bd9ab-24aa-49c8-9025-694a47d5ad5e None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:28.860841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0fb67-ea34-467b-9a2f-9193a6bd6e4d None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:28.880126 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:28.905655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b4d20-c147-4e2d-bf81-a8f26c8fc735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:28.906450 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b4d20-c147-4e2d-bf81-a8f26c8fc735 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:28.906740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b4d20-c147-4e2d-bf81-a8f26c8fc735 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:28.907022 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b4d20-c147-4e2d-bf81-a8f26c8fc735 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:28.908061 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b4d20-c147-4e2d-bf81-a8f26c8fc735 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:28.908377 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b4d20-c147-4e2d-bf81-a8f26c8fc735 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:28.910761 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b4d20-c147-4e2d-bf81-a8f26c8fc735 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:28.921850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 112/222] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:28.927324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b4d20-c147-4e2d-bf81-a8f26c8fc735 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:28.947043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bd9ab-24aa-49c8-9025-694a47d5ad5e None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:28.991392 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29252722-17a2-49bb-8295-8163bcd2a660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:28.992535 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29252722-17a2-49bb-8295-8163bcd2a660 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:28.992818 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29252722-17a2-49bb-8295-8163bcd2a660 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:28.993119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29252722-17a2-49bb-8295-8163bcd2a660 None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:28.994681 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29252722-17a2-49bb-8295-8163bcd2a660 None admin] RBAC: Authorizing `identity:get_project(project_id=956f270e083e4b15b70833346dcb8e19)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:28.996707 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29252722-17a2-49bb-8295-8163bcd2a660 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:28.999595 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 111/223] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:28 2022] GET /identity/v3/projects/956f270e083e4b15b70833346dcb8e19 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:29.004769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49317acf-66c6-479d-bdb6-55d1c263c780 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:29.004769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49317acf-66c6-479d-bdb6-55d1c263c780 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:29.004769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49317acf-66c6-479d-bdb6-55d1c263c780 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:29.004769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 112/224] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:29.010466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49317acf-66c6-479d-bdb6-55d1c263c780 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:29.059552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:29.069911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b4d20-c147-4e2d-bf81-a8f26c8fc735 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:29.116751 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515c1c2-3ae8-475b-8153-b5b8e9869adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:29.117730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515c1c2-3ae8-475b-8153-b5b8e9869adf None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:29.118016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515c1c2-3ae8-475b-8153-b5b8e9869adf None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:29.118294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515c1c2-3ae8-475b-8153-b5b8e9869adf None admin] PATH_INFO: `/v3/users/81e7e312cce349e8b47efbbeb888c186` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:29.119860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515c1c2-3ae8-475b-8153-b5b8e9869adf None admin] RBAC: Authorizing `identity:get_user(user_id=81e7e312cce349e8b47efbbeb888c186)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:29.122148 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515c1c2-3ae8-475b-8153-b5b8e9869adf None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:29.124768 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 113/225] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:28 2022] GET /identity/v3/users/81e7e312cce349e8b47efbbeb888c186 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:29.126420 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edad3ae-1eda-47b5-bffe-68c86d36894b None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:29.126707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edad3ae-1eda-47b5-bffe-68c86d36894b None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:29.126983 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edad3ae-1eda-47b5-bffe-68c86d36894b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:29.143978 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49317acf-66c6-479d-bdb6-55d1c263c780 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:29.144744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-7edad3ae-1eda-47b5-bffe-68c86d36894b None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:29.146223 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-7edad3ae-1eda-47b5-bffe-68c86d36894b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:29.160289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:29.184129 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c0951-b5bc-47b0-8573-016095c35ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:29.185694 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c0951-b5bc-47b0-8573-016095c35ff0 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:29.186221 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c0951-b5bc-47b0-8573-016095c35ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:29.186594 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c0951-b5bc-47b0-8573-016095c35ff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:29.187599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c0951-b5bc-47b0-8573-016095c35ff0 None admin] RBAC: Adding query filter params (role.id=8a0eeb2a21dc4897a58fb5851d58678c, scope.project.id=956f270e083e4b15b70833346dcb8e19, user.id=ef5705177006412fb518485c2913ee25) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:29.188067 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c0951-b5bc-47b0-8573-016095c35ff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:29.189887 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c0951-b5bc-47b0-8573-016095c35ff0 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:29.195862 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 113/226] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:25:29 2022] GET /identity/v3/role_assignments?user.id=ef5705177006412fb518485c2913ee25&scope.project.id=956f270e083e4b15b70833346dcb8e19&role.id=8a0eeb2a21dc4897a58fb5851d58678c => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:29.202282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c0951-b5bc-47b0-8573-016095c35ff0 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:29.227926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 114/227] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:29.260668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edad3ae-1eda-47b5-bffe-68c86d36894b None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:29.378328 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c0951-b5bc-47b0-8573-016095c35ff0 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:29.381628 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:29.398590 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edad3ae-1eda-47b5-bffe-68c86d36894b None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:29.421639 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd51df-d07b-4ae9-bd9a-e3706090fbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:29.421639 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd51df-d07b-4ae9-bd9a-e3706090fbac None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:29.421639 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd51df-d07b-4ae9-bd9a-e3706090fbac None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:29.421639 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd51df-d07b-4ae9-bd9a-e3706090fbac None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:29.422109 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd51df-d07b-4ae9-bd9a-e3706090fbac None admin] RBAC: Authorizing `identity:get_project(project_id=956f270e083e4b15b70833346dcb8e19)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:29.422109 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd51df-d07b-4ae9-bd9a-e3706090fbac None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:29.423951 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 114/228] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:29 2022] GET /identity/v3/projects/956f270e083e4b15b70833346dcb8e19 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:29.434381 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd51df-d07b-4ae9-bd9a-e3706090fbac None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:29.452312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180ea34-908d-44dc-806b-b3d7ae68acf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:29.453744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180ea34-908d-44dc-806b-b3d7ae68acf0 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:29.454242 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180ea34-908d-44dc-806b-b3d7ae68acf0 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:29.455521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180ea34-908d-44dc-806b-b3d7ae68acf0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:29.468171 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180ea34-908d-44dc-806b-b3d7ae68acf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:29.477906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-a180ea34-908d-44dc-806b-b3d7ae68acf0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:29.477906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:29.477906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:29.477906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:29.477906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:29.477906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:29.477906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:29.477906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:29.477906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:29.477906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:29.477906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:29.477906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:29.477906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:29.477906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:29.477906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:29.477906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:29.478887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:29.478887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:29.478887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:29.478887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:29.478887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:29.478887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:29.478887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:29.478887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:29.478887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:29.478887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:29.478887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:29.478887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:29.478887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:29.478887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:29.478887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:29.478887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:29.479685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:29.479685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:29.479685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:29.479685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:29.479685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:29.479685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:29.479685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:29.479685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:29.479685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:29.479685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:29.479685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:29.479685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:29.483181 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 115/229] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:25:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:25:29.492438 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180ea34-908d-44dc-806b-b3d7ae68acf0 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:29.495425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:29.561259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd51df-d07b-4ae9-bd9a-e3706090fbac None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:29.600307 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f4c26-96f0-4176-ae38-dff3b39690bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:29.601486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f4c26-96f0-4176-ae38-dff3b39690bf None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:29.601777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f4c26-96f0-4176-ae38-dff3b39690bf None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:29.602058 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f4c26-96f0-4176-ae38-dff3b39690bf None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19/users/81e7e312cce349e8b47efbbeb888c186/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:29.604481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f4c26-96f0-4176-ae38-dff3b39690bf None admin] RBAC: Authorizing `identity:create_grant(project_id=956f270e083e4b15b70833346dcb8e19, user_id=81e7e312cce349e8b47efbbeb888c186, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:29.607365 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f4c26-96f0-4176-ae38-dff3b39690bf None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:29.623920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 115/230] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:25:29 2022] PUT /identity/v3/projects/956f270e083e4b15b70833346dcb8e19/users/81e7e312cce349e8b47efbbeb888c186/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:25:29.655461 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180ea34-908d-44dc-806b-b3d7ae68acf0 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:29.725471 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ddaa3-ba62-4bab-88f5-eddccc040ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:29.726432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ddaa3-ba62-4bab-88f5-eddccc040ecd None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:29.726858 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ddaa3-ba62-4bab-88f5-eddccc040ecd None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:29.727307 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ddaa3-ba62-4bab-88f5-eddccc040ecd None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:29.728224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ddaa3-ba62-4bab-88f5-eddccc040ecd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:29.728694 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ddaa3-ba62-4bab-88f5-eddccc040ecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:29.730347 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ddaa3-ba62-4bab-88f5-eddccc040ecd None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:29.745590 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 116/231] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:25:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:29.752070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f4c26-96f0-4176-ae38-dff3b39690bf None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:29.879227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f4c26-96f0-4176-ae38-dff3b39690bf None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:29.886913 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:29.930437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df04f2-13cd-4a92-962f-988f55d79ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:29.931304 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df04f2-13cd-4a92-962f-988f55d79ff9 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:29.931587 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df04f2-13cd-4a92-962f-988f55d79ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:29.931864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df04f2-13cd-4a92-962f-988f55d79ff9 None admin] PATH_INFO: `/v3/users/81e7e312cce349e8b47efbbeb888c186` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:29.933153 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df04f2-13cd-4a92-962f-988f55d79ff9 None admin] RBAC: Authorizing `identity:get_user(user_id=81e7e312cce349e8b47efbbeb888c186)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:29.934623 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df04f2-13cd-4a92-962f-988f55d79ff9 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:29.936841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 116/232] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:29 2022] GET /identity/v3/users/81e7e312cce349e8b47efbbeb888c186 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:29.945196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ddaa3-ba62-4bab-88f5-eddccc040ecd None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:29.973354 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:30.088270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ddaa3-ba62-4bab-88f5-eddccc040ecd None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:30.134040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03423d-230f-4495-b645-31d3224bf49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:30.135270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03423d-230f-4495-b645-31d3224bf49c None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:30.135753 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03423d-230f-4495-b645-31d3224bf49c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:30.136228 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03423d-230f-4495-b645-31d3224bf49c None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:30.141803 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03423d-230f-4495-b645-31d3224bf49c None admin] RBAC: Authorizing `identity:get_project(project_id=956f270e083e4b15b70833346dcb8e19)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:30.143486 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03423d-230f-4495-b645-31d3224bf49c None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:30.145424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 117/233] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:29 2022] GET /identity/v3/projects/956f270e083e4b15b70833346dcb8e19 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:30.151180 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df04f2-13cd-4a92-962f-988f55d79ff9 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:30.289439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df04f2-13cd-4a92-962f-988f55d79ff9 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:30.326355 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbc0a0-17b7-4c96-a71b-babad27103c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:30.327254 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbc0a0-17b7-4c96-a71b-babad27103c8 None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:30.327536 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbc0a0-17b7-4c96-a71b-babad27103c8 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:30.327809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbc0a0-17b7-4c96-a71b-babad27103c8 None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19/users/81e7e312cce349e8b47efbbeb888c186/roles/bad7164dd96e4048bab9360ef3f6769e` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:30.330115 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dbc0a0-17b7-4c96-a71b-babad27103c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=956f270e083e4b15b70833346dcb8e19, user_id=81e7e312cce349e8b47efbbeb888c186, role_id=bad7164dd96e4048bab9360ef3f6769e)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:30.331587 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dbc0a0-17b7-4c96-a71b-babad27103c8 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:30.347105 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 117/234] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:25:30 2022] PUT /identity/v3/projects/956f270e083e4b15b70833346dcb8e19/users/81e7e312cce349e8b47efbbeb888c186/roles/bad7164dd96e4048bab9360ef3f6769e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:25:30.960944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09151738-4d3c-4397-8ed8-1cbcc4461dc8 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:30.961514 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09151738-4d3c-4397-8ed8-1cbcc4461dc8 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:30.961514 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09151738-4d3c-4397-8ed8-1cbcc4461dc8 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:30.963239 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 118/235] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:30.968827 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559a288-9a0a-4169-b60a-be1936e4e363 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:30.969271 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559a288-9a0a-4169-b60a-be1936e4e363 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:30.969500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559a288-9a0a-4169-b60a-be1936e4e363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:30.993332 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-1559a288-9a0a-4169-b60a-be1936e4e363 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:30.994530 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-1559a288-9a0a-4169-b60a-be1936e4e363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:31.034965 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:31.077937 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac72e9-ce82-469d-84d3-c8f6897f4bb2 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:31.078246 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac72e9-ce82-469d-84d3-c8f6897f4bb2 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:31.078499 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac72e9-ce82-469d-84d3-c8f6897f4bb2 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:31.080040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 119/236] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:31.081573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 118/237] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:31.085737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ed2e6-2daf-4ae2-84c2-f9a05caf563c None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:31.086021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ed2e6-2daf-4ae2-84c2-f9a05caf563c None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:31.086303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ed2e6-2daf-4ae2-84c2-f9a05caf563c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:31.106807 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559a288-9a0a-4169-b60a-be1936e4e363 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:31.114301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-934ed2e6-2daf-4ae2-84c2-f9a05caf563c None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:31.115534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-934ed2e6-2daf-4ae2-84c2-f9a05caf563c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:31.250522 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 120/238] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:31.282702 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559a288-9a0a-4169-b60a-be1936e4e363 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:31.300487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ed2e6-2daf-4ae2-84c2-f9a05caf563c None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:31.317400 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:31.355175 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9514855-ff64-42d8-a272-2e50838d511a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:31.356113 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9514855-ff64-42d8-a272-2e50838d511a None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:31.356430 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9514855-ff64-42d8-a272-2e50838d511a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:31.356739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9514855-ff64-42d8-a272-2e50838d511a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:31.379518 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9514855-ff64-42d8-a272-2e50838d511a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:31.392215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-b9514855-ff64-42d8-a272-2e50838d511a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:25:31.392215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:31.392215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:31.392215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:31.392215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:31.392215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:31.392215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:31.392215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:31.392215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:31.392215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:31.392215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:31.392215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:31.392215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:31.392215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:31.392215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:31.392215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:31.393198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:31.393198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:31.393198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:31.393198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:31.393198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:31.393198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:31.393198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:31.393198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:31.393198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:31.393198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:31.393198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:31.393198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:31.393198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:31.393198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:31.393198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:31.393198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:31.394027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:31.394027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:31.394027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:31.394027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:31.394027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:31.394027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:31.394027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:31.394027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:31.394027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:31.394027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:31.394027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:25:31.394027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:31.394027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 119/239] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:25:31.403124 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9514855-ff64-42d8-a272-2e50838d511a None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:31.440936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:31.483464 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ed2e6-2daf-4ae2-84c2-f9a05caf563c None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:31.529925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ad9c4-cbbb-448f-aa66-776808f547b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:31.530657 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ad9c4-cbbb-448f-aa66-776808f547b2 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:31.530941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ad9c4-cbbb-448f-aa66-776808f547b2 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:31.531217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ad9c4-cbbb-448f-aa66-776808f547b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:31.534827 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ad9c4-cbbb-448f-aa66-776808f547b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:31.536561 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ad9c4-cbbb-448f-aa66-776808f547b2 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:31.538493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 121/240] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:25:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:31.545251 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ad9c4-cbbb-448f-aa66-776808f547b2 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:31.560350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9514855-ff64-42d8-a272-2e50838d511a None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:31.595439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604c1ba-cd78-4f60-a557-26f55155c17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:31.596122 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604c1ba-cd78-4f60-a557-26f55155c17d None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:31.596791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604c1ba-cd78-4f60-a557-26f55155c17d None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:31.597087 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604c1ba-cd78-4f60-a557-26f55155c17d None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:31.597863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604c1ba-cd78-4f60-a557-26f55155c17d None admin] RBAC: Adding query filter params (name=reader) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:31.598173 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604c1ba-cd78-4f60-a557-26f55155c17d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:31.599853 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604c1ba-cd78-4f60-a557-26f55155c17d None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:31.609025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 120/241] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:31.614382 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604c1ba-cd78-4f60-a557-26f55155c17d None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:31.745435 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:31.754508 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:31.769939 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604c1ba-cd78-4f60-a557-26f55155c17d None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:31.793928 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ad9c4-cbbb-448f-aa66-776808f547b2 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:31.821892 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce69c4-3468-42b0-ab08-5f2d46424ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:31.822780 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce69c4-3468-42b0-ab08-5f2d46424ba6 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:31.823063 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce69c4-3468-42b0-ab08-5f2d46424ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:31.823336 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce69c4-3468-42b0-ab08-5f2d46424ba6 None admin] PATH_INFO: `/v3/users/461a62d3e6bb4f6fbf2442410f3b873f` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:31.839338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce69c4-3468-42b0-ab08-5f2d46424ba6 None admin] RBAC: Authorizing `identity:get_user(user_id=461a62d3e6bb4f6fbf2442410f3b873f)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:31.840817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce69c4-3468-42b0-ab08-5f2d46424ba6 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:31.843487 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 121/242] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:31 2022] GET /identity/v3/users/461a62d3e6bb4f6fbf2442410f3b873f => generated 297 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:31.848715 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce69c4-3468-42b0-ab08-5f2d46424ba6 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:31.884260 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b885a7-ac74-4c5c-97e4-9a5008b738a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:31.886020 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b885a7-ac74-4c5c-97e4-9a5008b738a9 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:31.886305 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b885a7-ac74-4c5c-97e4-9a5008b738a9 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:31.886581 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b885a7-ac74-4c5c-97e4-9a5008b738a9 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:31.889235 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b885a7-ac74-4c5c-97e4-9a5008b738a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:31.891551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b885a7-ac74-4c5c-97e4-9a5008b738a9 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:31.932907 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 122/243] 10.222.0.71 () {60 vars in 1132 bytes} [Thu Jul 21 20:25:31 2022] POST /identity/v3/users => generated 293 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:25:32.048462 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce69c4-3468-42b0-ab08-5f2d46424ba6 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:32.050978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec29e13-9add-4241-8722-fc56aacc1bf4 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:32.051441 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec29e13-9add-4241-8722-fc56aacc1bf4 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:32.051544 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec29e13-9add-4241-8722-fc56aacc1bf4 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:32.054444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 123/244] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:25:32.060460 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871169d6-dfeb-42a3-a18a-94b9d99579f7 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:32.061032 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871169d6-dfeb-42a3-a18a-94b9d99579f7 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:32.061553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871169d6-dfeb-42a3-a18a-94b9d99579f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:32.093189 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-871169d6-dfeb-42a3-a18a-94b9d99579f7 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:32.094490 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-871169d6-dfeb-42a3-a18a-94b9d99579f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:32.104948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aec93f-33ba-4bc4-a2c1-3a55025be969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:32.105955 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aec93f-33ba-4bc4-a2c1-3a55025be969 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:32.107329 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aec93f-33ba-4bc4-a2c1-3a55025be969 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:32.107799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aec93f-33ba-4bc4-a2c1-3a55025be969 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:32.141463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:32.177590 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 124/245] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:32.179723 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17168c-da6d-4ff1-85e3-4319b90d0fb2 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:32.180191 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17168c-da6d-4ff1-85e3-4319b90d0fb2 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:32.180617 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17168c-da6d-4ff1-85e3-4319b90d0fb2 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:32.186531 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 125/246] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:32.190959 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aec93f-33ba-4bc4-a2c1-3a55025be969 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:32.192462 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aec93f-33ba-4bc4-a2c1-3a55025be969 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:32.192631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd8285-b88b-4459-8fd7-068af32ca1d4 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:32.193140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd8285-b88b-4459-8fd7-068af32ca1d4 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:32.193564 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd8285-b88b-4459-8fd7-068af32ca1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:32.213465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-cdcd8285-b88b-4459-8fd7-068af32ca1d4 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:32.214885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-cdcd8285-b88b-4459-8fd7-068af32ca1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:32.291950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:32.299321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 126/247] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:32.300563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd8285-b88b-4459-8fd7-068af32ca1d4 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:32.313115 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 122/248] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:32.319687 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aec93f-33ba-4bc4-a2c1-3a55025be969 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:32.378808 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:32.432307 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd8285-b88b-4459-8fd7-068af32ca1d4 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:32.448499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aec93f-33ba-4bc4-a2c1-3a55025be969 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:32.487719 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b255f-1c83-4339-99d0-72251ccaf4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:32.489505 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b255f-1c83-4339-99d0-72251ccaf4d2 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:32.490229 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b255f-1c83-4339-99d0-72251ccaf4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:32.490718 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b255f-1c83-4339-99d0-72251ccaf4d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:32.506465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b255f-1c83-4339-99d0-72251ccaf4d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:32.507438 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3bfd21-a39c-4ec4-b269-5a0dbca884df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:32.508510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bfd21-a39c-4ec4-b269-5a0dbca884df None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:32.508989 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bfd21-a39c-4ec4-b269-5a0dbca884df None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:32.509469 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bfd21-a39c-4ec4-b269-5a0dbca884df None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:32.510877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3bfd21-a39c-4ec4-b269-5a0dbca884df None admin] RBAC: Authorizing `identity:get_project(project_id=956f270e083e4b15b70833346dcb8e19)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:32.512509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3bfd21-a39c-4ec4-b269-5a0dbca884df None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:32.515298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 123/249] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:32 2022] GET /identity/v3/projects/956f270e083e4b15b70833346dcb8e19 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:32.515439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-a85b255f-1c83-4339-99d0-72251ccaf4d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:32.515439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:32.515439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:32.515439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:32.515439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:32.515439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:32.515439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:32.515439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:32.515439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:32.515439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:32.515439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:32.515439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:32.515439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:32.515439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:32.515439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:32.515439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:32.516300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:32.516300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:32.516300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:32.516300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:32.516300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:32.516300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:32.516300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:32.516300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:32.516300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:32.516300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:32.516300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:32.516300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:32.516300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:32.516300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:32.516300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:32.516300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:32.521184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:32.521184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:32.521184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:32.521184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:32.521184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:32.521184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:32.521184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:32.521184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:32.521184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:32.521184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:32.521184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:32.521184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:32.521184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 127/250] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:32.521184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b255f-1c83-4339-99d0-72251ccaf4d2 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:32.523208 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3bfd21-a39c-4ec4-b269-5a0dbca884df None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:32.592693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:32.655438 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3bfd21-a39c-4ec4-b269-5a0dbca884df None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:32.671175 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:32.696767 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01355d-bf62-4c7e-a3dd-6dc34a111773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:32.697734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01355d-bf62-4c7e-a3dd-6dc34a111773 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:32.698510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01355d-bf62-4c7e-a3dd-6dc34a111773 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:32.699099 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01355d-bf62-4c7e-a3dd-6dc34a111773 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:32.700233 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d01355d-bf62-4c7e-a3dd-6dc34a111773 None admin] RBAC: Adding query filter params (role.id=bc82ed55215d47b791d5aa25699e7e2b, scope.project.id=956f270e083e4b15b70833346dcb8e19, user.id=461a62d3e6bb4f6fbf2442410f3b873f) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:32.700760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d01355d-bf62-4c7e-a3dd-6dc34a111773 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:32.702601 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d01355d-bf62-4c7e-a3dd-6dc34a111773 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:32.709307 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 124/251] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:25:32 2022] GET /identity/v3/role_assignments?user.id=461a62d3e6bb4f6fbf2442410f3b873f&scope.project.id=956f270e083e4b15b70833346dcb8e19&role.id=bc82ed55215d47b791d5aa25699e7e2b => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:32.710669 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01355d-bf62-4c7e-a3dd-6dc34a111773 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:32.723238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b255f-1c83-4339-99d0-72251ccaf4d2 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:32.838355 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01355d-bf62-4c7e-a3dd-6dc34a111773 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:32.850217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44796123-5a0a-45e1-bab9-3dfd4ca85a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:32.851058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44796123-5a0a-45e1-bab9-3dfd4ca85a9f None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:32.851417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44796123-5a0a-45e1-bab9-3dfd4ca85a9f None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:32.851771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44796123-5a0a-45e1-bab9-3dfd4ca85a9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:32.894111 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44796123-5a0a-45e1-bab9-3dfd4ca85a9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:32.902661 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2895e-7d0c-4323-82b2-ed803590cf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:32.903690 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2895e-7d0c-4323-82b2-ed803590cf9b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:32.904182 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2895e-7d0c-4323-82b2-ed803590cf9b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:32.904466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-44796123-5a0a-45e1-bab9-3dfd4ca85a9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:32.904466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:32.904466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:32.904466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:32.904466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:32.904466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:32.904466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:32.904466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:32.904466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:32.904466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:32.904466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:32.904466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:32.904466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:32.904466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:32.904466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:32.904466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:32.909886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:32.909886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:32.909886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:32.909886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:32.909886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:32.909886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:32.909886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:32.909886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:32.909886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:32.909886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:32.909886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:32.909886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:32.909886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:32.909886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:32.909886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:32.909886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:32.910689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:32.910689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:32.910689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:32.910689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:32.910689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:32.910689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:32.910689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:32.910689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:32.910689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:32.910689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:32.910689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:32.910689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:32.910689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 128/252] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:25:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:32.912117 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2895e-7d0c-4323-82b2-ed803590cf9b None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:32.913200 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2895e-7d0c-4323-82b2-ed803590cf9b None admin] RBAC: Adding query filter params (name=member) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:32.913697 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2895e-7d0c-4323-82b2-ed803590cf9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:32.915289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2895e-7d0c-4323-82b2-ed803590cf9b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:32.925182 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e219f2-e8e7-4253-9d7f-fd4501c9bd00 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:32.925474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e219f2-e8e7-4253-9d7f-fd4501c9bd00 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:32.925750 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e219f2-e8e7-4253-9d7f-fd4501c9bd00 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:32.927119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 129/253] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:25:32.930084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 125/254] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:32.932114 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9d7c7-cd8f-40e6-aca0-b49923b43596 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:32.935295 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9d7c7-cd8f-40e6-aca0-b49923b43596 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:32.935295 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9d7c7-cd8f-40e6-aca0-b49923b43596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:32.936536 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2895e-7d0c-4323-82b2-ed803590cf9b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:32.951578 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-7ad9d7c7-cd8f-40e6-aca0-b49923b43596 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:32.953290 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-7ad9d7c7-cd8f-40e6-aca0-b49923b43596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:33.024929 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:33.057719 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 130/255] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:33.059252 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9d7c7-cd8f-40e6-aca0-b49923b43596 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:33.115107 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2895e-7d0c-4323-82b2-ed803590cf9b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:33.140426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:33.152825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400d528-7fea-4cc7-b1ce-5a732c7fbfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:33.153802 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400d528-7fea-4cc7-b1ce-5a732c7fbfc6 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:33.154084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400d528-7fea-4cc7-b1ce-5a732c7fbfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:33.154360 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400d528-7fea-4cc7-b1ce-5a732c7fbfc6 None admin] PATH_INFO: `/v3/users/ef5705177006412fb518485c2913ee25` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:33.158975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400d528-7fea-4cc7-b1ce-5a732c7fbfc6 None admin] RBAC: Authorizing `identity:get_user(user_id=ef5705177006412fb518485c2913ee25)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:33.158975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400d528-7fea-4cc7-b1ce-5a732c7fbfc6 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:33.159304 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 126/256] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:32 2022] GET /identity/v3/users/ef5705177006412fb518485c2913ee25 => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:33.160033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400d528-7fea-4cc7-b1ce-5a732c7fbfc6 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:33.241278 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9d7c7-cd8f-40e6-aca0-b49923b43596 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:33.264890 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400d528-7fea-4cc7-b1ce-5a732c7fbfc6 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:33.274701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:33.303201 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a67cc-af0f-4488-a853-a8cff0af71e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:33.304092 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a67cc-af0f-4488-a853-a8cff0af71e4 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:33.304458 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a67cc-af0f-4488-a853-a8cff0af71e4 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:33.304733 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a67cc-af0f-4488-a853-a8cff0af71e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:33.313953 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611480be-e36a-4619-8707-aab5cdbabe66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:33.314651 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611480be-e36a-4619-8707-aab5cdbabe66 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:33.314935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611480be-e36a-4619-8707-aab5cdbabe66 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:33.315211 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611480be-e36a-4619-8707-aab5cdbabe66 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:33.315737 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a67cc-af0f-4488-a853-a8cff0af71e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:33.316002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611480be-e36a-4619-8707-aab5cdbabe66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:33.316319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611480be-e36a-4619-8707-aab5cdbabe66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:33.317745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611480be-e36a-4619-8707-aab5cdbabe66 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:33.324046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-414a67cc-af0f-4488-a853-a8cff0af71e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:33.324046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:33.324046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:33.324046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:33.324046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:33.324046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:33.324046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:33.324046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:33.324046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:33.324046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:33.324046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:33.324046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:33.324046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:33.324046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:33.324046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:33.324046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:33.324966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:33.324966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:33.324966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:33.324966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:33.324966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:33.324966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:33.324966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:33.324966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:33.324966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:33.324966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:33.324966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:33.324966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:33.324966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:33.324966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:33.324966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:33.324966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:33.326948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:33.326948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:33.326948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:33.326948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:33.326948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:33.326948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:33.326948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:33.326948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:33.326948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:33.326948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:33.326948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:33.326948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:33.326948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 127/257] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:25:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:33.326948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a67cc-af0f-4488-a853-a8cff0af71e4 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:33.341251 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 131/258] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:33.342215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611480be-e36a-4619-8707-aab5cdbabe66 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:33.438772 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a67cc-af0f-4488-a853-a8cff0af71e4 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:33.481486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9f1c0-c350-460c-851d-9e189a8867ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:33.483007 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9f1c0-c350-460c-851d-9e189a8867ec None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:33.483658 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9f1c0-c350-460c-851d-9e189a8867ec None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:33.483939 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9f1c0-c350-460c-851d-9e189a8867ec None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:33.485330 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9f1c0-c350-460c-851d-9e189a8867ec None admin] RBAC: Authorizing `identity:get_project(project_id=956f270e083e4b15b70833346dcb8e19)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:33.487456 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9f1c0-c350-460c-851d-9e189a8867ec None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:33.489306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 128/259] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:33 2022] GET /identity/v3/projects/956f270e083e4b15b70833346dcb8e19 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:33.490273 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9f1c0-c350-460c-851d-9e189a8867ec None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:33.529521 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:33.603251 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9f1c0-c350-460c-851d-9e189a8867ec None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:33.640510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885e3fe-4813-4612-9155-37004275358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:33.641416 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885e3fe-4813-4612-9155-37004275358b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:33.641736 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885e3fe-4813-4612-9155-37004275358b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:33.642460 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885e3fe-4813-4612-9155-37004275358b None admin] PATH_INFO: `/v3/users/81e7e312cce349e8b47efbbeb888c186` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:33.643660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885e3fe-4813-4612-9155-37004275358b None admin] RBAC: Authorizing `identity:get_user(user_id=81e7e312cce349e8b47efbbeb888c186)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:33.645149 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885e3fe-4813-4612-9155-37004275358b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:33.647252 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 129/260] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:33 2022] GET /identity/v3/users/81e7e312cce349e8b47efbbeb888c186 => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:33.648144 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885e3fe-4813-4612-9155-37004275358b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:33.672979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611480be-e36a-4619-8707-aab5cdbabe66 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:33.691544 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:33.757383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885e3fe-4813-4612-9155-37004275358b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:33.764917 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:33.793338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea20bae-1d32-4c5f-a58f-275aaab5bb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:33.794371 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea20bae-1d32-4c5f-a58f-275aaab5bb96 None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:33.794713 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea20bae-1d32-4c5f-a58f-275aaab5bb96 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:33.795047 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea20bae-1d32-4c5f-a58f-275aaab5bb96 None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19/users/ef5705177006412fb518485c2913ee25/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:33.797215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea20bae-1d32-4c5f-a58f-275aaab5bb96 None admin] RBAC: Authorizing `identity:create_grant(project_id=956f270e083e4b15b70833346dcb8e19, user_id=ef5705177006412fb518485c2913ee25, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:33.798661 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea20bae-1d32-4c5f-a58f-275aaab5bb96 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:33.804574 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55bfac-c784-4fa5-9008-780ae8ff567f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:33.805458 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55bfac-c784-4fa5-9008-780ae8ff567f None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:33.805740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55bfac-c784-4fa5-9008-780ae8ff567f None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:33.806084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55bfac-c784-4fa5-9008-780ae8ff567f None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:33.806879 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55bfac-c784-4fa5-9008-780ae8ff567f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:33.807191 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55bfac-c784-4fa5-9008-780ae8ff567f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:33.808618 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55bfac-c784-4fa5-9008-780ae8ff567f None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:33.816021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 130/261] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:25:33 2022] PUT /identity/v3/projects/956f270e083e4b15b70833346dcb8e19/users/ef5705177006412fb518485c2913ee25/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:25:33.816860 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea20bae-1d32-4c5f-a58f-275aaab5bb96 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:33.836023 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 132/262] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:25:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 494 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:33.845447 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55bfac-c784-4fa5-9008-780ae8ff567f None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:34.023596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea20bae-1d32-4c5f-a58f-275aaab5bb96 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:34.063538 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9c737-093a-4959-b24b-83a6555cd29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:34.064216 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9c737-093a-4959-b24b-83a6555cd29d None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:34.064530 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9c737-093a-4959-b24b-83a6555cd29d None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:34.064802 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9c737-093a-4959-b24b-83a6555cd29d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:34.101497 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:34.119419 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55bfac-c784-4fa5-9008-780ae8ff567f None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:34.143963 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb9c737-093a-4959-b24b-83a6555cd29d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:34.145495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb9c737-093a-4959-b24b-83a6555cd29d None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:34.193825 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815e036-3a3f-4caa-a2c5-d0a939e06ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:34.194723 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815e036-3a3f-4caa-a2c5-d0a939e06ba6 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:34.195009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815e036-3a3f-4caa-a2c5-d0a939e06ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:34.195289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815e036-3a3f-4caa-a2c5-d0a939e06ba6 None admin] PATH_INFO: `/v3/users/81e7e312cce349e8b47efbbeb888c186` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:34.201887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815e036-3a3f-4caa-a2c5-d0a939e06ba6 None admin] RBAC: Authorizing `identity:get_user(user_id=81e7e312cce349e8b47efbbeb888c186)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:34.203345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815e036-3a3f-4caa-a2c5-d0a939e06ba6 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:34.205044 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 133/263] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:33 2022] GET /identity/v3/users/81e7e312cce349e8b47efbbeb888c186 => generated 283 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:34.214137 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815e036-3a3f-4caa-a2c5-d0a939e06ba6 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:34.260339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 131/264] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:34.265807 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9c737-093a-4959-b24b-83a6555cd29d None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:34.283617 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:34.349733 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:34.379791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9c737-093a-4959-b24b-83a6555cd29d None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:34.388409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815e036-3a3f-4caa-a2c5-d0a939e06ba6 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:34.417324 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9fd45-16d8-42a1-8bad-5bd2d9378315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:34.419081 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9fd45-16d8-42a1-8bad-5bd2d9378315 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:34.420155 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9fd45-16d8-42a1-8bad-5bd2d9378315 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:34.420572 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9fd45-16d8-42a1-8bad-5bd2d9378315 None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:34.421959 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9fd45-16d8-42a1-8bad-5bd2d9378315 None admin] RBAC: Authorizing `identity:get_project(project_id=956f270e083e4b15b70833346dcb8e19)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:34.423678 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9fd45-16d8-42a1-8bad-5bd2d9378315 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:34.425955 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 132/265] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:34 2022] GET /identity/v3/projects/956f270e083e4b15b70833346dcb8e19 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:34.434142 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9fd45-16d8-42a1-8bad-5bd2d9378315 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:34.465803 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac73c0b-ca19-443a-b985-042b35e90706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:34.467008 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac73c0b-ca19-443a-b985-042b35e90706 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:34.467295 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac73c0b-ca19-443a-b985-042b35e90706 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:34.467577 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac73c0b-ca19-443a-b985-042b35e90706 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:34.548183 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac73c0b-ca19-443a-b985-042b35e90706 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:34.549645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac73c0b-ca19-443a-b985-042b35e90706 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:34.591821 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:34.651010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9fd45-16d8-42a1-8bad-5bd2d9378315 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:34.685360 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:34.693746 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c33528-49e6-43d9-a485-143d16db93a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:34.694750 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c33528-49e6-43d9-a485-143d16db93a6 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:34.695182 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c33528-49e6-43d9-a485-143d16db93a6 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:34.695603 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c33528-49e6-43d9-a485-143d16db93a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:34.696800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c33528-49e6-43d9-a485-143d16db93a6 None admin] RBAC: Adding query filter params (role.id=19f5e72c0be44a1197c7288de82e79e8, scope.project.id=956f270e083e4b15b70833346dcb8e19, user.id=81e7e312cce349e8b47efbbeb888c186) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:34.697289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c33528-49e6-43d9-a485-143d16db93a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:34.699317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c33528-49e6-43d9-a485-143d16db93a6 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:34.706959 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 133/266] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:25:34 2022] GET /identity/v3/role_assignments?user.id=81e7e312cce349e8b47efbbeb888c186&scope.project.id=956f270e083e4b15b70833346dcb8e19&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:34.708659 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821d107-945c-4053-a00b-08e714006607 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:34.709203 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821d107-945c-4053-a00b-08e714006607 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:34.709745 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821d107-945c-4053-a00b-08e714006607 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:34.711246 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 134/267] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:34.718652 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3df7d0-7f99-49da-b9b4-43ea6777c9cc None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:34.719271 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3df7d0-7f99-49da-b9b4-43ea6777c9cc None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:34.720414 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3df7d0-7f99-49da-b9b4-43ea6777c9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:34.750306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-ee3df7d0-7f99-49da-b9b4-43ea6777c9cc None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:34.752039 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-ee3df7d0-7f99-49da-b9b4-43ea6777c9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:34.781580 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 134/268] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:34.790271 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac73c0b-ca19-443a-b985-042b35e90706 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:34.907089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 135/269] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:34.954624 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3df7d0-7f99-49da-b9b4-43ea6777c9cc None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:34.989100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac73c0b-ca19-443a-b985-042b35e90706 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:35.029380 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ec78c-b659-43c4-ae32-efbd33b04fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:35.030391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ec78c-b659-43c4-ae32-efbd33b04fe9 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:35.030669 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ec78c-b659-43c4-ae32-efbd33b04fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:35.030941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ec78c-b659-43c4-ae32-efbd33b04fe9 None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:35.032324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ec78c-b659-43c4-ae32-efbd33b04fe9 None admin] RBAC: Authorizing `identity:get_project(project_id=956f270e083e4b15b70833346dcb8e19)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:35.034231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ec78c-b659-43c4-ae32-efbd33b04fe9 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:35.036267 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 135/270] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:34 2022] GET /identity/v3/projects/956f270e083e4b15b70833346dcb8e19 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:35.041887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ec78c-b659-43c4-ae32-efbd33b04fe9 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:35.051372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:35.173811 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3df7d0-7f99-49da-b9b4-43ea6777c9cc None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:35.224053 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:35.229460 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ec78c-b659-43c4-ae32-efbd33b04fe9 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:35.251787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4288429-1b42-4711-aca8-1ab1f111f68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:35.252602 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4288429-1b42-4711-aca8-1ab1f111f68a None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:35.252887 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4288429-1b42-4711-aca8-1ab1f111f68a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:35.253188 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4288429-1b42-4711-aca8-1ab1f111f68a None admin] PATH_INFO: `/v3/roles/member` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:35.271310 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4288429-1b42-4711-aca8-1ab1f111f68a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:35.279219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-e4288429-1b42-4711-aca8-1ab1f111f68a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:35.279219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:35.279219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:35.279219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:35.279219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:35.279219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:35.279219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:35.279219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:35.279219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:35.279219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:35.279219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:35.279219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:35.279219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:35.279219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:35.279219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:35.279219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:35.280112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:35.280112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:35.280112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:35.280112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:35.280112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:35.280112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:35.280112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:35.280112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:35.280112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:35.280112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:35.280112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:35.280112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:35.280112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:35.280112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:35.280112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:35.280112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:35.280942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:35.280942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:35.280942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:35.280942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:35.280942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:35.280942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:35.280942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:35.280942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:35.280942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:35.280942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:35.280942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:35.280942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:35.280942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 136/271] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:35.284522 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e906b5-8d8a-434f-87ab-67740b9636f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:35.285444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e906b5-8d8a-434f-87ab-67740b9636f2 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:35.285565 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e906b5-8d8a-434f-87ab-67740b9636f2 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:35.285759 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e906b5-8d8a-434f-87ab-67740b9636f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:35.286602 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e906b5-8d8a-434f-87ab-67740b9636f2 None admin] RBAC: Adding query filter params (role.id=bad7164dd96e4048bab9360ef3f6769e, scope.project.id=956f270e083e4b15b70833346dcb8e19, user.id=81e7e312cce349e8b47efbbeb888c186) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:35.286921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e906b5-8d8a-434f-87ab-67740b9636f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:35.287326 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4288429-1b42-4711-aca8-1ab1f111f68a None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:35.288499 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e906b5-8d8a-434f-87ab-67740b9636f2 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:35.301238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 136/272] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:25:35 2022] GET /identity/v3/role_assignments?user.id=81e7e312cce349e8b47efbbeb888c186&scope.project.id=956f270e083e4b15b70833346dcb8e19&role.id=bad7164dd96e4048bab9360ef3f6769e => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:35.358796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1cd53-f409-4868-ac8c-becf2836130f None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:35.359146 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1cd53-f409-4868-ac8c-becf2836130f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:35.359359 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1cd53-f409-4868-ac8c-becf2836130f None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:35.360777 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 137/273] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:35.370630 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27805208-96f7-401f-8f8c-bc1a07c39e57 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:35.371073 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27805208-96f7-401f-8f8c-bc1a07c39e57 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:35.371358 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27805208-96f7-401f-8f8c-bc1a07c39e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:35.401724 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4288429-1b42-4711-aca8-1ab1f111f68a None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:35.410402 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-27805208-96f7-401f-8f8c-bc1a07c39e57 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:35.411649 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-27805208-96f7-401f-8f8c-bc1a07c39e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:35.449390 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1b3af-5f01-4693-bec9-5eea130877fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:35.449991 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1b3af-5f01-4693-bec9-5eea130877fc None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:35.450275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1b3af-5f01-4693-bec9-5eea130877fc None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:35.451145 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1b3af-5f01-4693-bec9-5eea130877fc None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:35.453556 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1b3af-5f01-4693-bec9-5eea130877fc None admin] RBAC: Adding query filter params (name=member) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:35.453874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1b3af-5f01-4693-bec9-5eea130877fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:35.455310 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1b3af-5f01-4693-bec9-5eea130877fc None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:35.476926 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 137/274] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:35.484067 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1b3af-5f01-4693-bec9-5eea130877fc None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:35.534881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:35.539933 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 138/275] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:35.548482 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:35.577860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27805208-96f7-401f-8f8c-bc1a07c39e57 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:35.618219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1b3af-5f01-4693-bec9-5eea130877fc None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:35.660435 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a9588-d6d5-4cb4-8879-c8a91990a187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:35.661677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a9588-d6d5-4cb4-8879-c8a91990a187 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:35.662040 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a9588-d6d5-4cb4-8879-c8a91990a187 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:35.662564 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a9588-d6d5-4cb4-8879-c8a91990a187 None admin] PATH_INFO: `/v3/users/ac6cc7d666fb41448525da784a87c4ec` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:35.679714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a9588-d6d5-4cb4-8879-c8a91990a187 None admin] RBAC: Authorizing `identity:get_user(user_id=ac6cc7d666fb41448525da784a87c4ec)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:35.681372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a9588-d6d5-4cb4-8879-c8a91990a187 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:35.683775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 138/276] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:35 2022] GET /identity/v3/users/ac6cc7d666fb41448525da784a87c4ec => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:35.691328 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a9588-d6d5-4cb4-8879-c8a91990a187 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:35.716113 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27805208-96f7-401f-8f8c-bc1a07c39e57 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:35.758608 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9deaa-1c01-468a-8166-d91ae7015e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:35.759408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9deaa-1c01-468a-8166-d91ae7015e61 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:35.759690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9deaa-1c01-468a-8166-d91ae7015e61 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:35.759964 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9deaa-1c01-468a-8166-d91ae7015e61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:35.761176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9deaa-1c01-468a-8166-d91ae7015e61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:35.762636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9deaa-1c01-468a-8166-d91ae7015e61 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:35.764994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 139/277] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:25:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:35.766112 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3649781-6333-479b-a22e-67c3077e77c2 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:35.766404 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3649781-6333-479b-a22e-67c3077e77c2 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:35.766679 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3649781-6333-479b-a22e-67c3077e77c2 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:35.768059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 140/278] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:35.772690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3649781-6333-479b-a22e-67c3077e77c2 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:35.809518 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a9588-d6d5-4cb4-8879-c8a91990a187 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:35.813369 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:35.845197 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:35.848152 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215b932-4b90-459a-bf6e-06c4e89431ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:35.848807 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215b932-4b90-459a-bf6e-06c4e89431ed None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:35.849410 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215b932-4b90-459a-bf6e-06c4e89431ed None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:35.849662 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215b932-4b90-459a-bf6e-06c4e89431ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:35.850520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215b932-4b90-459a-bf6e-06c4e89431ed None admin] RBAC: Adding query filter params (role.id=8a0eeb2a21dc4897a58fb5851d58678c, user.id=ac6cc7d666fb41448525da784a87c4ec, scope.system=all) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:35.850822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215b932-4b90-459a-bf6e-06c4e89431ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:35.852249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215b932-4b90-459a-bf6e-06c4e89431ed None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:35.857727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 139/279] 10.222.0.71 () {56 vars in 1308 bytes} [Thu Jul 21 20:25:35 2022] GET /identity/v3/role_assignments?user.id=ac6cc7d666fb41448525da784a87c4ec&scope.system=all&role.id=8a0eeb2a21dc4897a58fb5851d58678c => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:35.858697 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9337a2e-0614-48c7-9638-6f296aff4453 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:35.858980 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9337a2e-0614-48c7-9638-6f296aff4453 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:35.859258 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9337a2e-0614-48c7-9638-6f296aff4453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:35.885248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-e9337a2e-0614-48c7-9638-6f296aff4453 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:35.886138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3649781-6333-479b-a22e-67c3077e77c2 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:35.886641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-e9337a2e-0614-48c7-9638-6f296aff4453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:35.928108 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36af11-2884-4b03-ab36-0c6c7a8ba58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:35.929175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36af11-2884-4b03-ab36-0c6c7a8ba58c None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:35.929639 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36af11-2884-4b03-ab36-0c6c7a8ba58c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:35.930130 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36af11-2884-4b03-ab36-0c6c7a8ba58c None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:35.931079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36af11-2884-4b03-ab36-0c6c7a8ba58c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:35.932909 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36af11-2884-4b03-ab36-0c6c7a8ba58c None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:35.960623 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 141/280] 10.222.0.71 () {60 vars in 1132 bytes} [Thu Jul 21 20:25:35 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:25:35.985530 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 140/281] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:36.012584 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36af11-2884-4b03-ab36-0c6c7a8ba58c None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:36.127237 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36af11-2884-4b03-ab36-0c6c7a8ba58c None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:36.154261 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:36.167719 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331af012-9b94-457d-b641-56c2a3355e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:36.168520 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331af012-9b94-457d-b641-56c2a3355e7a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:36.168799 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331af012-9b94-457d-b641-56c2a3355e7a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:36.169082 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331af012-9b94-457d-b641-56c2a3355e7a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:36.182805 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331af012-9b94-457d-b641-56c2a3355e7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:36.190611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-331af012-9b94-457d-b641-56c2a3355e7a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:25:36.190611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:36.190611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:36.190611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:36.190611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:36.190611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:36.190611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:36.190611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:36.190611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:36.190611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:36.190611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:36.190611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:36.190611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:36.190611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:36.190611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:36.190611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:36.191512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:36.191512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:36.191512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:36.191512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:36.191512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:36.191512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:36.191512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:36.191512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:36.191512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:36.191512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:36.191512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:36.191512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:36.191512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:36.191512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:36.191512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:36.191512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:36.192298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:36.192298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:36.192298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:36.192298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:36.192298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:36.192298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:36.192298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:36.192298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:36.192298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:36.192298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:36.192298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:25:36.192298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:36.192298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 142/282] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:36.197836 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9337a2e-0614-48c7-9638-6f296aff4453 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:36.307194 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:36.311443 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9337a2e-0614-48c7-9638-6f296aff4453 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:36.351697 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860bbb8b-1217-4e8c-8c96-ae3d1ed5d169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:36.352416 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860bbb8b-1217-4e8c-8c96-ae3d1ed5d169 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:36.352697 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860bbb8b-1217-4e8c-8c96-ae3d1ed5d169 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:36.352974 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860bbb8b-1217-4e8c-8c96-ae3d1ed5d169 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:36.353783 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860bbb8b-1217-4e8c-8c96-ae3d1ed5d169 None admin] RBAC: Adding query filter params (name=reader) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:36.354091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860bbb8b-1217-4e8c-8c96-ae3d1ed5d169 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:36.355467 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860bbb8b-1217-4e8c-8c96-ae3d1ed5d169 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:36.365304 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 141/283] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:36.371064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331af012-9b94-457d-b641-56c2a3355e7a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:36.485741 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331af012-9b94-457d-b641-56c2a3355e7a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:36.536678 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd548435-58cb-4b80-acaf-0f4bc5778fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:36.537590 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd548435-58cb-4b80-acaf-0f4bc5778fd9 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:36.537872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd548435-58cb-4b80-acaf-0f4bc5778fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:36.538145 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd548435-58cb-4b80-acaf-0f4bc5778fd9 None admin] PATH_INFO: `/v3/users/461a62d3e6bb4f6fbf2442410f3b873f` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:36.539379 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd548435-58cb-4b80-acaf-0f4bc5778fd9 None admin] RBAC: Authorizing `identity:get_user(user_id=461a62d3e6bb4f6fbf2442410f3b873f)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:36.540859 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd548435-58cb-4b80-acaf-0f4bc5778fd9 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:36.542521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 143/284] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:36 2022] GET /identity/v3/users/461a62d3e6bb4f6fbf2442410f3b873f => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:36.548283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860bbb8b-1217-4e8c-8c96-ae3d1ed5d169 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:36.559287 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f76c6-08df-460c-aa92-9a84cdbe9ded None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:36.559553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f76c6-08df-460c-aa92-9a84cdbe9ded None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:36.559798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f76c6-08df-460c-aa92-9a84cdbe9ded None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:36.562209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 144/285] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:36.568689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986aa4e-5adb-48c1-aa19-c898638f2c22 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:36.568978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986aa4e-5adb-48c1-aa19-c898638f2c22 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:36.569283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986aa4e-5adb-48c1-aa19-c898638f2c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:36.594404 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-4986aa4e-5adb-48c1-aa19-c898638f2c22 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:36.595636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-4986aa4e-5adb-48c1-aa19-c898638f2c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:36.662926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:36.693560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 145/286] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:36.724655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986aa4e-5adb-48c1-aa19-c898638f2c22 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:36.735293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860bbb8b-1217-4e8c-8c96-ae3d1ed5d169 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:36.800440 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3c4c7-ceb0-4e97-9c83-4754c80e30f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:36.801367 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3c4c7-ceb0-4e97-9c83-4754c80e30f4 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:36.801653 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3c4c7-ceb0-4e97-9c83-4754c80e30f4 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:36.801932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3c4c7-ceb0-4e97-9c83-4754c80e30f4 None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:36.803262 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3c4c7-ceb0-4e97-9c83-4754c80e30f4 None admin] RBAC: Authorizing `identity:get_project(project_id=956f270e083e4b15b70833346dcb8e19)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:36.804753 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3c4c7-ceb0-4e97-9c83-4754c80e30f4 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:36.806918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 142/287] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:36 2022] GET /identity/v3/projects/956f270e083e4b15b70833346dcb8e19 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:36.812785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3c4c7-ceb0-4e97-9c83-4754c80e30f4 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:36.870498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:36.899173 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986aa4e-5adb-48c1-aa19-c898638f2c22 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:36.987258 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3c4c7-ceb0-4e97-9c83-4754c80e30f4 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:36.989256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0640c-af11-4e7c-bf99-44edfe807fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:36.990066 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0640c-af11-4e7c-bf99-44edfe807fbf None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:36.990346 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0640c-af11-4e7c-bf99-44edfe807fbf None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:36.990615 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0640c-af11-4e7c-bf99-44edfe807fbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:37.013811 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b0640c-af11-4e7c-bf99-44edfe807fbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:37.026952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-f7b0640c-af11-4e7c-bf99-44edfe807fbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:37.026952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:37.026952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:37.026952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:37.026952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:37.026952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:37.026952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:37.026952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:37.026952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:37.026952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:37.026952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:37.026952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:37.026952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:37.026952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:37.026952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:37.026952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:37.027882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:37.027882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:37.027882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:37.027882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:37.027882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:37.027882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:37.027882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:37.027882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:37.027882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:37.027882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:37.027882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:37.027882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:37.027882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:37.027882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:37.027882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:37.027882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:37.028800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:37.028800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:37.028800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:37.028800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:37.028800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:37.028800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:37.028800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:37.028800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:37.028800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:37.028800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:37.028800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:37.028800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:37.033091 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 146/288] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:37.034986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0640c-af11-4e7c-bf99-44edfe807fbf None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:37.053514 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4999c7-fcf9-44f3-abc6-7313db0d7241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:37.054471 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4999c7-fcf9-44f3-abc6-7313db0d7241 None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:37.054760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4999c7-fcf9-44f3-abc6-7313db0d7241 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:37.055040 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4999c7-fcf9-44f3-abc6-7313db0d7241 None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19/users/461a62d3e6bb4f6fbf2442410f3b873f/roles/bc82ed55215d47b791d5aa25699e7e2b` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:37.057821 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4999c7-fcf9-44f3-abc6-7313db0d7241 None admin] RBAC: Authorizing `identity:create_grant(project_id=956f270e083e4b15b70833346dcb8e19, user_id=461a62d3e6bb4f6fbf2442410f3b873f, role_id=bc82ed55215d47b791d5aa25699e7e2b)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:37.059285 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4999c7-fcf9-44f3-abc6-7313db0d7241 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:37.085827 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 143/289] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:25:36 2022] PUT /identity/v3/projects/956f270e083e4b15b70833346dcb8e19/users/461a62d3e6bb4f6fbf2442410f3b873f/roles/bc82ed55215d47b791d5aa25699e7e2b => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:25:37.123446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:37.207456 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0640c-af11-4e7c-bf99-44edfe807fbf None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:37.250567 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e51d47-da74-49fd-94ad-05f9c9c87d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:37.251612 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e51d47-da74-49fd-94ad-05f9c9c87d85 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:37.252117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e51d47-da74-49fd-94ad-05f9c9c87d85 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:37.252588 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e51d47-da74-49fd-94ad-05f9c9c87d85 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:37.253558 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e51d47-da74-49fd-94ad-05f9c9c87d85 None admin] RBAC: Adding query filter params (name=member) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:37.254062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e51d47-da74-49fd-94ad-05f9c9c87d85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:37.255636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e51d47-da74-49fd-94ad-05f9c9c87d85 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:37.266969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 147/290] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:37.272286 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4999c7-fcf9-44f3-abc6-7313db0d7241 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:37.376409 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:37.388862 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4999c7-fcf9-44f3-abc6-7313db0d7241 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:37.425681 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fffd2c-1943-4ffb-9839-64e4eef47cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:37.426551 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fffd2c-1943-4ffb-9839-64e4eef47cd1 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:37.426831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fffd2c-1943-4ffb-9839-64e4eef47cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:37.427105 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fffd2c-1943-4ffb-9839-64e4eef47cd1 None admin] PATH_INFO: `/v3/users/ef5705177006412fb518485c2913ee25` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:37.428310 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fffd2c-1943-4ffb-9839-64e4eef47cd1 None admin] RBAC: Authorizing `identity:get_user(user_id=ef5705177006412fb518485c2913ee25)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:37.429774 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fffd2c-1943-4ffb-9839-64e4eef47cd1 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:37.431753 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 144/291] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:37 2022] GET /identity/v3/users/ef5705177006412fb518485c2913ee25 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:37.442153 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e51d47-da74-49fd-94ad-05f9c9c87d85 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:37.581961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e51d47-da74-49fd-94ad-05f9c9c87d85 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:37.586504 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:37.629752 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fcbc6-22f4-411a-8a56-d6c1d9b0e324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:37.630317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fcbc6-22f4-411a-8a56-d6c1d9b0e324 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:37.630597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fcbc6-22f4-411a-8a56-d6c1d9b0e324 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:37.630867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fcbc6-22f4-411a-8a56-d6c1d9b0e324 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:37.727778 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fcbc6-22f4-411a-8a56-d6c1d9b0e324 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:37.729352 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fcbc6-22f4-411a-8a56-d6c1d9b0e324 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:37.871677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 148/292] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:37.877506 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fffd2c-1943-4ffb-9839-64e4eef47cd1 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:37.983079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fffd2c-1943-4ffb-9839-64e4eef47cd1 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:38.017845 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d815f-6b8b-40aa-93f8-9cd63d5b6149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:38.018715 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d815f-6b8b-40aa-93f8-9cd63d5b6149 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:38.018994 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d815f-6b8b-40aa-93f8-9cd63d5b6149 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:38.019265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d815f-6b8b-40aa-93f8-9cd63d5b6149 None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:38.020475 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d815f-6b8b-40aa-93f8-9cd63d5b6149 None admin] RBAC: Authorizing `identity:get_project(project_id=956f270e083e4b15b70833346dcb8e19)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:38.021946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d815f-6b8b-40aa-93f8-9cd63d5b6149 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:38.023634 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 145/293] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:37 2022] GET /identity/v3/projects/956f270e083e4b15b70833346dcb8e19 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:38.030320 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fcbc6-22f4-411a-8a56-d6c1d9b0e324 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:38.033476 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:38.164270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fcbc6-22f4-411a-8a56-d6c1d9b0e324 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:38.215631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324f8a4-f095-499d-8e92-a199a1ccaae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:38.216832 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324f8a4-f095-499d-8e92-a199a1ccaae7 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:38.217137 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324f8a4-f095-499d-8e92-a199a1ccaae7 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:38.217415 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324f8a4-f095-499d-8e92-a199a1ccaae7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:38.218256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324f8a4-f095-499d-8e92-a199a1ccaae7 None admin] RBAC: Adding query filter params (role.id=8a0eeb2a21dc4897a58fb5851d58678c, scope.project.id=956f270e083e4b15b70833346dcb8e19, user.id=ef5705177006412fb518485c2913ee25) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:38.218567 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324f8a4-f095-499d-8e92-a199a1ccaae7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:38.220021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324f8a4-f095-499d-8e92-a199a1ccaae7 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:38.229881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 149/294] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:25:38 2022] GET /identity/v3/role_assignments?user.id=ef5705177006412fb518485c2913ee25&scope.project.id=956f270e083e4b15b70833346dcb8e19&role.id=8a0eeb2a21dc4897a58fb5851d58678c => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:38.433896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47661577-03e0-4b15-bb75-b1fb4f062117 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:38.434421 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47661577-03e0-4b15-bb75-b1fb4f062117 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:38.434421 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47661577-03e0-4b15-bb75-b1fb4f062117 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:38.436119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 146/295] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:38.442188 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54176f7e-b2d0-457e-8c48-2d89f0828dc1 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:38.443190 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54176f7e-b2d0-457e-8c48-2d89f0828dc1 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:38.443696 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54176f7e-b2d0-457e-8c48-2d89f0828dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:38.480581 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-54176f7e-b2d0-457e-8c48-2d89f0828dc1 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:38.482331 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-54176f7e-b2d0-457e-8c48-2d89f0828dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:38.565490 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:38.577173 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 150/296] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:38.595853 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47661577-03e0-4b15-bb75-b1fb4f062117 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:38.626565 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:38.674415 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4990383-1ed8-470c-9e21-bb70c29f937b None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:38.675092 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4990383-1ed8-470c-9e21-bb70c29f937b None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:38.675591 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4990383-1ed8-470c-9e21-bb70c29f937b None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:38.677634 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 151/297] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:25:38.683547 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1e46a-b72e-499f-a025-e7e656f4e887 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:38.684269 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1e46a-b72e-499f-a025-e7e656f4e887 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:38.685323 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1e46a-b72e-499f-a025-e7e656f4e887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:38.706435 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-2cd1e46a-b72e-499f-a025-e7e656f4e887 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:38.708841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-2cd1e46a-b72e-499f-a025-e7e656f4e887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:38.712363 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47661577-03e0-4b15-bb75-b1fb4f062117 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:38.753575 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a0c48-59a3-4b4b-b28d-fb6bd98a2086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:38.754733 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a0c48-59a3-4b4b-b28d-fb6bd98a2086 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:38.755181 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a0c48-59a3-4b4b-b28d-fb6bd98a2086 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:38.755662 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a0c48-59a3-4b4b-b28d-fb6bd98a2086 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:38.778024 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a0c48-59a3-4b4b-b28d-fb6bd98a2086 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:38.790953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-c14a0c48-59a3-4b4b-b28d-fb6bd98a2086 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:38.790953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:38.790953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:38.790953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:38.790953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:38.790953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:38.790953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:38.790953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:38.790953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:38.790953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:38.790953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:38.790953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:38.790953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:38.790953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:38.790953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:38.790953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:38.793250 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:38.793250 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:38.793250 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:38.793250 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:38.793250 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:38.793250 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:38.793250 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:38.793250 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:38.793250 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:38.793250 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:38.793250 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:38.793250 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:38.793250 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:38.793250 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:38.793250 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:38.793250 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:38.794065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:38.794065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:38.794065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:38.794065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:38.794065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:38.794065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:38.794065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:38.794065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:38.794065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:38.794065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:38.794065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:38.794065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:38.794065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 147/298] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:38.794065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3eaef-035a-4a96-9681-02ed210c70a4 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:38.794849 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3eaef-035a-4a96-9681-02ed210c70a4 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:38.794849 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3eaef-035a-4a96-9681-02ed210c70a4 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:38.796078 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 148/299] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:38.799371 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 152/300] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:38.800975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3eaef-035a-4a96-9681-02ed210c70a4 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:38.806748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e84342-7ba6-4054-b666-f75e042f6900 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:38.807040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e84342-7ba6-4054-b666-f75e042f6900 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:38.807336 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e84342-7ba6-4054-b666-f75e042f6900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:38.837015 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-07e84342-7ba6-4054-b666-f75e042f6900 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:38.838925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-07e84342-7ba6-4054-b666-f75e042f6900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:38.899723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:38.917169 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 153/301] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:38.918650 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e84342-7ba6-4054-b666-f75e042f6900 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:38.929364 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3eaef-035a-4a96-9681-02ed210c70a4 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:38.975583 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705d64d-c87b-4dbb-af03-61e7023b7160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:38.976175 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705d64d-c87b-4dbb-af03-61e7023b7160 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:38.976460 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705d64d-c87b-4dbb-af03-61e7023b7160 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:38.976737 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705d64d-c87b-4dbb-af03-61e7023b7160 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:38.977542 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705d64d-c87b-4dbb-af03-61e7023b7160 None admin] RBAC: Adding query filter params (name=member) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:38.977851 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705d64d-c87b-4dbb-af03-61e7023b7160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:38.979291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705d64d-c87b-4dbb-af03-61e7023b7160 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:38.984187 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:38.989348 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 149/302] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:38.994646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705d64d-c87b-4dbb-af03-61e7023b7160 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:39.031858 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e84342-7ba6-4054-b666-f75e042f6900 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:39.067591 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b309d7-8e85-4e71-86e7-474bd5b6dfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:39.068389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b309d7-8e85-4e71-86e7-474bd5b6dfa1 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:39.068668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b309d7-8e85-4e71-86e7-474bd5b6dfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:39.068944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b309d7-8e85-4e71-86e7-474bd5b6dfa1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:39.079891 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b309d7-8e85-4e71-86e7-474bd5b6dfa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:39.087534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-a0b309d7-8e85-4e71-86e7-474bd5b6dfa1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:25:39.087534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:39.087534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:39.087534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:39.087534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:39.087534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:39.087534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:39.087534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:39.087534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:39.087534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:39.087534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:39.087534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:39.087534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:39.087534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:39.087534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:39.087534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:39.088443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:39.088443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:39.088443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:39.088443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:39.088443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:39.088443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:39.088443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:39.088443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:39.088443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:39.088443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:39.088443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:39.088443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:39.088443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:39.088443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:39.088443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:39.088443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:39.093836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:39.093836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:39.093836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:39.093836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:39.093836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:39.093836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:39.093836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:39.093836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:39.093836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:39.093836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:39.093836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:25:39.093836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:39.093836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 154/303] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:39.093836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b309d7-8e85-4e71-86e7-474bd5b6dfa1 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:39.109786 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705d64d-c87b-4dbb-af03-61e7023b7160 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:39.146580 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a84be-0922-4ef3-a7f9-3e4ed696f1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:39.147366 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a84be-0922-4ef3-a7f9-3e4ed696f1c1 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:39.147643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a84be-0922-4ef3-a7f9-3e4ed696f1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:39.148030 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a84be-0922-4ef3-a7f9-3e4ed696f1c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:39.149230 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a84be-0922-4ef3-a7f9-3e4ed696f1c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:39.150676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a84be-0922-4ef3-a7f9-3e4ed696f1c1 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:39.152546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 150/304] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:25:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:39.153042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a84be-0922-4ef3-a7f9-3e4ed696f1c1 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:39.170103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:39.203529 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b309d7-8e85-4e71-86e7-474bd5b6dfa1 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:39.221459 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:39.239873 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d6e2e-9590-422f-9822-16c77a766b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:39.240663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d6e2e-9590-422f-9822-16c77a766b1e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:39.240942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d6e2e-9590-422f-9822-16c77a766b1e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:39.241311 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d6e2e-9590-422f-9822-16c77a766b1e None admin] PATH_INFO: `/v3/users/ac6cc7d666fb41448525da784a87c4ec` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:39.242520 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d6e2e-9590-422f-9822-16c77a766b1e None admin] RBAC: Authorizing `identity:get_user(user_id=ac6cc7d666fb41448525da784a87c4ec)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:39.244006 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d6e2e-9590-422f-9822-16c77a766b1e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:39.245675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 155/305] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:39 2022] GET /identity/v3/users/ac6cc7d666fb41448525da784a87c4ec => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:39.250599 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d6e2e-9590-422f-9822-16c77a766b1e None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:39.269884 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a84be-0922-4ef3-a7f9-3e4ed696f1c1 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:39.305603 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf375c8-6e7d-4146-baaf-da0a91af8219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:39.307075 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf375c8-6e7d-4146-baaf-da0a91af8219 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:39.307575 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf375c8-6e7d-4146-baaf-da0a91af8219 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:39.307952 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf375c8-6e7d-4146-baaf-da0a91af8219 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:39.308839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf375c8-6e7d-4146-baaf-da0a91af8219 None admin] RBAC: Adding query filter params (name=reader) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:39.309224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf375c8-6e7d-4146-baaf-da0a91af8219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:39.310657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf375c8-6e7d-4146-baaf-da0a91af8219 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:39.324740 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 151/306] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:39.324740 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf375c8-6e7d-4146-baaf-da0a91af8219 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:39.360034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d6e2e-9590-422f-9822-16c77a766b1e None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:39.393689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da791cd9-7b9f-457b-8528-91ad1fe916ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:39.394328 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da791cd9-7b9f-457b-8528-91ad1fe916ca None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:39.394607 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da791cd9-7b9f-457b-8528-91ad1fe916ca None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:39.394883 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da791cd9-7b9f-457b-8528-91ad1fe916ca None admin] PATH_INFO: `/v3/groups` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:39.395569 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da791cd9-7b9f-457b-8528-91ad1fe916ca None admin] RBAC: Authorizing `identity:create_group()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:39.397011 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da791cd9-7b9f-457b-8528-91ad1fe916ca None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:39.407708 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:39.421116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 156/307] 10.222.0.71 () {60 vars in 1134 bytes} [Thu Jul 21 20:25:39 2022] POST /identity/v3/groups => generated 229 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:25:39.421116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da791cd9-7b9f-457b-8528-91ad1fe916ca None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:39.449461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf375c8-6e7d-4146-baaf-da0a91af8219 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:39.478734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:39.510565 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1f533-b067-426b-94f0-c28c94cae8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:39.511424 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1f533-b067-426b-94f0-c28c94cae8c1 None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:39.511644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1f533-b067-426b-94f0-c28c94cae8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:39.511916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1f533-b067-426b-94f0-c28c94cae8c1 None admin] PATH_INFO: `/v3/system/users/ac6cc7d666fb41448525da784a87c4ec/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:39.513641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1f533-b067-426b-94f0-c28c94cae8c1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ac6cc7d666fb41448525da784a87c4ec, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:39.515065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1f533-b067-426b-94f0-c28c94cae8c1 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:39.526170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 152/308] 10.222.0.71 () {58 vars in 1334 bytes} [Thu Jul 21 20:25:39 2022] PUT /identity/v3/system/users/ac6cc7d666fb41448525da784a87c4ec/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:25:39.557593 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da791cd9-7b9f-457b-8528-91ad1fe916ca None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:39.598015 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e98fce-97d0-43f8-9c44-2c3d36641624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:39.598912 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e98fce-97d0-43f8-9c44-2c3d36641624 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:39.599444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e98fce-97d0-43f8-9c44-2c3d36641624 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:39.599714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e98fce-97d0-43f8-9c44-2c3d36641624 None admin] PATH_INFO: `/v3/users/514cebea3b6f46c5840a10f95b2c3a9c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:39.616393 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e98fce-97d0-43f8-9c44-2c3d36641624 None admin] RBAC: Authorizing `identity:get_user(user_id=514cebea3b6f46c5840a10f95b2c3a9c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:39.618005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e98fce-97d0-43f8-9c44-2c3d36641624 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:39.620144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 157/309] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:39 2022] GET /identity/v3/users/514cebea3b6f46c5840a10f95b2c3a9c => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:39.626146 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1f533-b067-426b-94f0-c28c94cae8c1 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:39.633423 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9174b82-f098-472d-ac60-d79b1e54319f None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:39.634031 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9174b82-f098-472d-ac60-d79b1e54319f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:39.634504 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9174b82-f098-472d-ac60-d79b1e54319f None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:39.636555 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 158/310] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:39.644121 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00001e0-e9f0-4b70-9ec5-ca5f6e086a2d None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:39.646279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00001e0-e9f0-4b70-9ec5-ca5f6e086a2d None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:39.646638 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00001e0-e9f0-4b70-9ec5-ca5f6e086a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:39.669761 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-b00001e0-e9f0-4b70-9ec5-ca5f6e086a2d None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:39.671286 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-b00001e0-e9f0-4b70-9ec5-ca5f6e086a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:39.755798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 159/311] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:39.761005 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1f533-b067-426b-94f0-c28c94cae8c1 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:39.779834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00001e0-e9f0-4b70-9ec5-ca5f6e086a2d None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:39.800008 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e55a6-5239-42bc-99be-72b7c09b4927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:39.800670 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e55a6-5239-42bc-99be-72b7c09b4927 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:39.800951 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e55a6-5239-42bc-99be-72b7c09b4927 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:39.801261 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e55a6-5239-42bc-99be-72b7c09b4927 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:39.802107 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e55a6-5239-42bc-99be-72b7c09b4927 None admin] RBAC: Adding query filter params (role.id=bc82ed55215d47b791d5aa25699e7e2b, user.id=514cebea3b6f46c5840a10f95b2c3a9c, scope.system=all) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:39.802516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e55a6-5239-42bc-99be-72b7c09b4927 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:39.803983 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e55a6-5239-42bc-99be-72b7c09b4927 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:39.810749 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 153/312] 10.222.0.71 () {56 vars in 1308 bytes} [Thu Jul 21 20:25:39 2022] GET /identity/v3/role_assignments?user.id=514cebea3b6f46c5840a10f95b2c3a9c&scope.system=all&role.id=bc82ed55215d47b791d5aa25699e7e2b => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:39.821728 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:39.900717 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00001e0-e9f0-4b70-9ec5-ca5f6e086a2d None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:39.939891 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472a4d0-0de2-4f3a-9399-74429eda73f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:39.941120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472a4d0-0de2-4f3a-9399-74429eda73f9 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:39.941634 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472a4d0-0de2-4f3a-9399-74429eda73f9 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:39.942110 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472a4d0-0de2-4f3a-9399-74429eda73f9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:39.955638 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472a4d0-0de2-4f3a-9399-74429eda73f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:39.963924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-b472a4d0-0de2-4f3a-9399-74429eda73f9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:25:39.963924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:39.963924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:39.963924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:39.963924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:39.963924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:39.963924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:39.963924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:39.963924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:39.963924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:39.963924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:39.963924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:39.963924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:39.963924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:39.963924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:39.963924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:39.965312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:39.965312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:39.965312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:39.965312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:39.965312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:39.965312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:39.965312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:39.965312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:39.965312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:39.965312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:39.965312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:39.965312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:39.965312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:39.965312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:39.965312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:39.965312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:39.966867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:39.966867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:39.966867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:39.966867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:39.966867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:39.966867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:39.966867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:39.966867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:39.966867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:39.966867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:39.966867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:25:39.966867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:39.967613 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 160/313] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:25:39.973914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e55a6-5239-42bc-99be-72b7c09b4927 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:39.980485 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:40.088021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e55a6-5239-42bc-99be-72b7c09b4927 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:40.123426 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc6a23-47f0-4efd-9d82-78680a37ffcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:40.124105 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc6a23-47f0-4efd-9d82-78680a37ffcb None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:40.124385 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc6a23-47f0-4efd-9d82-78680a37ffcb None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:40.124661 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc6a23-47f0-4efd-9d82-78680a37ffcb None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:40.125458 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc6a23-47f0-4efd-9d82-78680a37ffcb None admin] RBAC: Adding query filter params (name=reader) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:40.125768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc6a23-47f0-4efd-9d82-78680a37ffcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:40.127201 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc6a23-47f0-4efd-9d82-78680a37ffcb None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:40.136384 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 154/314] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:40.142441 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472a4d0-0de2-4f3a-9399-74429eda73f9 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:40.248631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:40.255641 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472a4d0-0de2-4f3a-9399-74429eda73f9 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:40.290842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d7be2-58ed-48e1-89f8-c67636c66e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:40.291710 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d7be2-58ed-48e1-89f8-c67636c66e8b None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:40.291984 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d7be2-58ed-48e1-89f8-c67636c66e8b None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:40.292254 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d7be2-58ed-48e1-89f8-c67636c66e8b None admin] PATH_INFO: `/v3/users/461a62d3e6bb4f6fbf2442410f3b873f` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:40.293484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d7be2-58ed-48e1-89f8-c67636c66e8b None admin] RBAC: Authorizing `identity:get_user(user_id=461a62d3e6bb4f6fbf2442410f3b873f)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:40.295134 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d7be2-58ed-48e1-89f8-c67636c66e8b None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:40.297087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 161/315] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:40 2022] GET /identity/v3/users/461a62d3e6bb4f6fbf2442410f3b873f => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:40.302786 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc6a23-47f0-4efd-9d82-78680a37ffcb None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:40.371263 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:40.407989 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc6a23-47f0-4efd-9d82-78680a37ffcb None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:40.443183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51634257-4d40-4bbb-8d0d-7da8ebcc779e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:40.443846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51634257-4d40-4bbb-8d0d-7da8ebcc779e None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:40.444134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51634257-4d40-4bbb-8d0d-7da8ebcc779e None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:40.444411 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51634257-4d40-4bbb-8d0d-7da8ebcc779e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:40.524362 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51634257-4d40-4bbb-8d0d-7da8ebcc779e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:40.525887 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51634257-4d40-4bbb-8d0d-7da8ebcc779e None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:40.591064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:40.629635 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 155/316] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:40.634991 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d7be2-58ed-48e1-89f8-c67636c66e8b None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:40.744310 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d7be2-58ed-48e1-89f8-c67636c66e8b None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:40.780287 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff85f06-179c-4af8-bfd8-e390ff10ffaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:40.781186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff85f06-179c-4af8-bfd8-e390ff10ffaf None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:40.781478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff85f06-179c-4af8-bfd8-e390ff10ffaf None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:40.781757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff85f06-179c-4af8-bfd8-e390ff10ffaf None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:40.782978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff85f06-179c-4af8-bfd8-e390ff10ffaf None admin] RBAC: Authorizing `identity:get_project(project_id=956f270e083e4b15b70833346dcb8e19)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:40.784541 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff85f06-179c-4af8-bfd8-e390ff10ffaf None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:40.786943 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 162/317] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:40 2022] GET /identity/v3/projects/956f270e083e4b15b70833346dcb8e19 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:40.794689 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51634257-4d40-4bbb-8d0d-7da8ebcc779e None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:40.897997 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51634257-4d40-4bbb-8d0d-7da8ebcc779e None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:40.936824 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47066f32-6272-4d38-9e73-a694cdd87a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:40.937616 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47066f32-6272-4d38-9e73-a694cdd87a8a None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:40.937898 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47066f32-6272-4d38-9e73-a694cdd87a8a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:40.938172 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47066f32-6272-4d38-9e73-a694cdd87a8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:40.939061 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47066f32-6272-4d38-9e73-a694cdd87a8a None admin] RBAC: Adding query filter params (role.id=bc82ed55215d47b791d5aa25699e7e2b, scope.project.id=956f270e083e4b15b70833346dcb8e19, user.id=461a62d3e6bb4f6fbf2442410f3b873f) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:40.939377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47066f32-6272-4d38-9e73-a694cdd87a8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:40.940922 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47066f32-6272-4d38-9e73-a694cdd87a8a None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:40.948293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 156/318] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:25:40 2022] GET /identity/v3/role_assignments?user.id=461a62d3e6bb4f6fbf2442410f3b873f&scope.project.id=956f270e083e4b15b70833346dcb8e19&role.id=bc82ed55215d47b791d5aa25699e7e2b => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:41.725184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b2736-73be-4de8-ac62-9b783b964fc5 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:41.725700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b2736-73be-4de8-ac62-9b783b964fc5 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:41.725700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b2736-73be-4de8-ac62-9b783b964fc5 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:41.727126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 163/319] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:41.732874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7a48a-3b5a-4149-a5ee-c74f1882e111 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:41.733315 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7a48a-3b5a-4149-a5ee-c74f1882e111 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:41.733484 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7a48a-3b5a-4149-a5ee-c74f1882e111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:41.738882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba27db9-f245-4d15-bb28-2a68859ec26b None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:41.739190 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba27db9-f245-4d15-bb28-2a68859ec26b None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:41.739565 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba27db9-f245-4d15-bb28-2a68859ec26b None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:41.740923 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 164/320] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:25:41.746296 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970b359-19cc-4420-94a6-dd2618978768 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:41.746596 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970b359-19cc-4420-94a6-dd2618978768 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:41.746866 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970b359-19cc-4420-94a6-dd2618978768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:41.751127 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-c4a7a48a-3b5a-4149-a5ee-c74f1882e111 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:41.752317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c4a7a48a-3b5a-4149-a5ee-c74f1882e111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:41.765785 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-c970b359-19cc-4420-94a6-dd2618978768 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:41.767180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-c970b359-19cc-4420-94a6-dd2618978768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:41.779840 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:41.793675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:41.831605 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 157/321] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:41.845816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 165/322] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:41.856389 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7a48a-3b5a-4149-a5ee-c74f1882e111 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:41.873041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970b359-19cc-4420-94a6-dd2618978768 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:41.970103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7a48a-3b5a-4149-a5ee-c74f1882e111 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:41.977497 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970b359-19cc-4420-94a6-dd2618978768 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:42.006382 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8c194-c9d1-412e-ac88-54ba0a90bb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:42.007172 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8c194-c9d1-412e-ac88-54ba0a90bb5b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:42.007448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8c194-c9d1-412e-ac88-54ba0a90bb5b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:42.007716 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8c194-c9d1-412e-ac88-54ba0a90bb5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:42.009022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8c194-c9d1-412e-ac88-54ba0a90bb5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:42.010479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8c194-c9d1-412e-ac88-54ba0a90bb5b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:42.012084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 158/323] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:25:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:42.013276 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5e893-785a-47e3-b67f-6693be63d066 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:42.013391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e646423-ee22-4c2c-b8e1-aea185faa134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:42.013555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5e893-785a-47e3-b67f-6693be63d066 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:42.013829 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5e893-785a-47e3-b67f-6693be63d066 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:42.014676 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e646423-ee22-4c2c-b8e1-aea185faa134 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:42.019673 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e646423-ee22-4c2c-b8e1-aea185faa134 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:42.019837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 159/324] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:42.019932 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e646423-ee22-4c2c-b8e1-aea185faa134 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:42.020017 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5e893-785a-47e3-b67f-6693be63d066 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:42.026750 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e646423-ee22-4c2c-b8e1-aea185faa134 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:42.035906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-6e646423-ee22-4c2c-b8e1-aea185faa134 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:42.035906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:42.035906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:42.035906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:42.035906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:42.035906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:42.035906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:42.035906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:42.035906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:42.035906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:42.035906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:42.035906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:42.035906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:42.035906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:42.035906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:42.035906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:42.036919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:42.036919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:42.036919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:42.036919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:42.036919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:42.036919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:42.036919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:42.036919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:42.036919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:42.036919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:42.036919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:42.036919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:42.036919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:42.036919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:42.036919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:42.036919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:42.037770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:42.037770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:42.037770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:42.037770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:42.037770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:42.037770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:42.037770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:42.037770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:42.037770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:42.037770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:42.037770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:42.037770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:42.037770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 166/325] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:42.038457 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cb841-1527-46d1-83fd-5b718d136068 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:42.043672 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cb841-1527-46d1-83fd-5b718d136068 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:42.043672 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cb841-1527-46d1-83fd-5b718d136068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:42.046636 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:42.059440 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-6f3cb841-1527-46d1-83fd-5b718d136068 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:42.060981 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-6f3cb841-1527-46d1-83fd-5b718d136068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:42.076406 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:42.122500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5e893-785a-47e3-b67f-6693be63d066 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:42.131050 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 167/326] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:42.132176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cb841-1527-46d1-83fd-5b718d136068 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:42.159838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3a08c-6f3a-400c-9c96-696babc2bf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:42.160514 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3a08c-6f3a-400c-9c96-696babc2bf88 None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:42.160799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3a08c-6f3a-400c-9c96-696babc2bf88 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:42.161200 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3a08c-6f3a-400c-9c96-696babc2bf88 None admin] PATH_INFO: `/v3/groups` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:42.161906 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3a08c-6f3a-400c-9c96-696babc2bf88 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:42.163449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3a08c-6f3a-400c-9c96-696babc2bf88 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:42.179047 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 160/327] 10.222.0.71 () {60 vars in 1134 bytes} [Thu Jul 21 20:25:42 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:25:42.179879 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3a08c-6f3a-400c-9c96-696babc2bf88 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:42.256617 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cb841-1527-46d1-83fd-5b718d136068 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:42.293294 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:42.298498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daffb57-7f5c-44b1-b720-01a0d3596135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:42.299443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daffb57-7f5c-44b1-b720-01a0d3596135 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:42.299954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daffb57-7f5c-44b1-b720-01a0d3596135 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:42.300569 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daffb57-7f5c-44b1-b720-01a0d3596135 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:42.301659 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daffb57-7f5c-44b1-b720-01a0d3596135 None admin] RBAC: Adding query filter params (name=member) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:42.301966 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daffb57-7f5c-44b1-b720-01a0d3596135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:42.303410 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daffb57-7f5c-44b1-b720-01a0d3596135 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:42.309432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3a08c-6f3a-400c-9c96-696babc2bf88 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:42.321547 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 168/328] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:42.322546 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daffb57-7f5c-44b1-b720-01a0d3596135 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:42.344158 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:42.355219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614c42a-a790-450a-8b88-a26d4d3a90a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:42.356140 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614c42a-a790-450a-8b88-a26d4d3a90a0 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:42.356383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614c42a-a790-450a-8b88-a26d4d3a90a0 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:42.356684 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614c42a-a790-450a-8b88-a26d4d3a90a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:42.370580 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614c42a-a790-450a-8b88-a26d4d3a90a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:42.378609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-2614c42a-a790-450a-8b88-a26d4d3a90a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:25:42.378609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:42.378609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:42.378609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:42.378609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:42.378609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:42.378609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:42.378609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:42.378609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:42.378609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:42.378609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:42.378609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:42.378609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:42.378609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:42.378609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:42.378609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:42.379502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:42.379502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:42.379502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:42.379502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:42.379502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:42.379502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:42.379502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:42.379502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:42.379502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:42.379502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:42.379502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:42.379502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:42.379502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:42.379502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:42.379502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:42.379502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:42.380283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:42.380283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:42.380283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:42.380283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:42.380283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:42.380283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:42.380283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:42.380283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:42.380283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:42.380283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:42.380283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:25:42.380283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:42.380283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 161/329] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:42.385589 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614c42a-a790-450a-8b88-a26d4d3a90a0 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:42.457860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daffb57-7f5c-44b1-b720-01a0d3596135 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:42.498305 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869f313-ff16-4a18-9f35-73aa2c597a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:42.499926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869f313-ff16-4a18-9f35-73aa2c597a7e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:42.499926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869f313-ff16-4a18-9f35-73aa2c597a7e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:42.499926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869f313-ff16-4a18-9f35-73aa2c597a7e None admin] PATH_INFO: `/v3/users/ac6cc7d666fb41448525da784a87c4ec` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:42.501120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869f313-ff16-4a18-9f35-73aa2c597a7e None admin] RBAC: Authorizing `identity:get_user(user_id=ac6cc7d666fb41448525da784a87c4ec)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:42.502565 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869f313-ff16-4a18-9f35-73aa2c597a7e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:42.504210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 169/330] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:42 2022] GET /identity/v3/users/ac6cc7d666fb41448525da784a87c4ec => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:42.510277 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869f313-ff16-4a18-9f35-73aa2c597a7e None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:42.522100 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614c42a-a790-450a-8b88-a26d4d3a90a0 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:42.567804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d86b3-3374-4405-84fe-374e81170481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:42.568516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d86b3-3374-4405-84fe-374e81170481 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:42.568793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d86b3-3374-4405-84fe-374e81170481 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:42.569089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d86b3-3374-4405-84fe-374e81170481 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:42.569847 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d86b3-3374-4405-84fe-374e81170481 None admin] RBAC: Adding query filter params (name=reader) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:42.570150 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d86b3-3374-4405-84fe-374e81170481 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:42.572742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d86b3-3374-4405-84fe-374e81170481 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:42.582335 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 162/331] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:42.587688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d86b3-3374-4405-84fe-374e81170481 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:42.612033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:42.624083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:42.649300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869f313-ff16-4a18-9f35-73aa2c597a7e None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:42.697714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbf6ff-65f4-40e5-9d3c-409e159eecd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:42.698377 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbf6ff-65f4-40e5-9d3c-409e159eecd7 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:42.698658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbf6ff-65f4-40e5-9d3c-409e159eecd7 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:42.699388 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbf6ff-65f4-40e5-9d3c-409e159eecd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:42.700233 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbf6ff-65f4-40e5-9d3c-409e159eecd7 None admin] RBAC: Adding query filter params (role.id=8a0eeb2a21dc4897a58fb5851d58678c, user.id=ac6cc7d666fb41448525da784a87c4ec, scope.system=all) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:42.701030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbf6ff-65f4-40e5-9d3c-409e159eecd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:42.702808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbf6ff-65f4-40e5-9d3c-409e159eecd7 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:42.722613 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 170/332] 10.222.0.71 () {56 vars in 1308 bytes} [Thu Jul 21 20:25:42 2022] GET /identity/v3/role_assignments?user.id=ac6cc7d666fb41448525da784a87c4ec&scope.system=all&role.id=8a0eeb2a21dc4897a58fb5851d58678c => generated 510 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:42.732685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d86b3-3374-4405-84fe-374e81170481 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:42.773380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4dc45-07d8-41d0-8021-d126648dac47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:42.774230 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4dc45-07d8-41d0-8021-d126648dac47 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:42.774514 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4dc45-07d8-41d0-8021-d126648dac47 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:42.774792 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4dc45-07d8-41d0-8021-d126648dac47 None admin] PATH_INFO: `/v3/users/514cebea3b6f46c5840a10f95b2c3a9c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:42.776345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4dc45-07d8-41d0-8021-d126648dac47 None admin] RBAC: Authorizing `identity:get_user(user_id=514cebea3b6f46c5840a10f95b2c3a9c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:42.777878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4dc45-07d8-41d0-8021-d126648dac47 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:42.779625 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 163/333] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:42 2022] GET /identity/v3/users/514cebea3b6f46c5840a10f95b2c3a9c => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:42.785543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbf6ff-65f4-40e5-9d3c-409e159eecd7 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:42.900903 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbf6ff-65f4-40e5-9d3c-409e159eecd7 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:42.938107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5d60b-6ba5-4a3f-b45f-f4107596e7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:42.939293 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5d60b-6ba5-4a3f-b45f-f4107596e7ee None admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:42.939763 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5d60b-6ba5-4a3f-b45f-f4107596e7ee None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:42.940321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5d60b-6ba5-4a3f-b45f-f4107596e7ee None admin] PATH_INFO: `/v3/system/users/514cebea3b6f46c5840a10f95b2c3a9c/roles/bc82ed55215d47b791d5aa25699e7e2b` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:42.942207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5d60b-6ba5-4a3f-b45f-f4107596e7ee None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=514cebea3b6f46c5840a10f95b2c3a9c, role_id=bc82ed55215d47b791d5aa25699e7e2b)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:42.943858 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5d60b-6ba5-4a3f-b45f-f4107596e7ee None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:42.955553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 171/334] 10.222.0.71 () {58 vars in 1334 bytes} [Thu Jul 21 20:25:42 2022] PUT /identity/v3/system/users/514cebea3b6f46c5840a10f95b2c3a9c/roles/bc82ed55215d47b791d5aa25699e7e2b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:25:44.526142 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b22cfe-b732-4df5-961a-a2d8cdcd693b None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:44.527154 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b22cfe-b732-4df5-961a-a2d8cdcd693b None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:44.527693 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b22cfe-b732-4df5-961a-a2d8cdcd693b None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:44.529641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 164/335] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:44.536041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b47fa-d808-4271-81fe-7a50b8ae0272 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:44.536918 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b47fa-d808-4271-81fe-7a50b8ae0272 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:44.537448 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b47fa-d808-4271-81fe-7a50b8ae0272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:44.543505 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:44.560642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-196b47fa-d808-4271-81fe-7a50b8ae0272 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:44.562231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-196b47fa-d808-4271-81fe-7a50b8ae0272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:44.649462 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 172/336] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:44.686595 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b22cfe-b732-4df5-961a-a2d8cdcd693b None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:44.781764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:44.811374 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b22cfe-b732-4df5-961a-a2d8cdcd693b None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:44.855931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a64bd4-f038-4a68-9c8c-f09e54ab5ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:44.857323 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a64bd4-f038-4a68-9c8c-f09e54ab5ab6 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:44.857621 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a64bd4-f038-4a68-9c8c-f09e54ab5ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:44.857898 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a64bd4-f038-4a68-9c8c-f09e54ab5ab6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:44.868631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a64bd4-f038-4a68-9c8c-f09e54ab5ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:44.876459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-75a64bd4-f038-4a68-9c8c-f09e54ab5ab6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:44.876459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:44.876459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:44.876459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:44.876459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:44.876459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:44.876459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:44.876459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:44.876459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:44.876459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:44.876459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:44.876459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:44.876459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:44.876459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:44.876459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:44.876459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:44.877437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:44.877437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:44.877437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:44.877437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:44.877437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:44.877437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:44.877437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:44.877437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:44.877437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:44.877437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:44.877437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:44.877437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:44.877437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:44.877437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:44.877437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:44.877437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:44.878269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:44.878269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:44.878269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:44.878269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:44.878269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:44.878269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:44.878269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:44.878269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:44.878269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:44.878269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:44.878269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:44.878269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:44.878269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 165/337] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:44.885633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b47fa-d808-4271-81fe-7a50b8ae0272 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:45.000139 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b47fa-d808-4271-81fe-7a50b8ae0272 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:45.037185 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:45.042475 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4eb970-352c-4d60-a97a-62d77d67d7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:45.043436 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4eb970-352c-4d60-a97a-62d77d67d7a6 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:45.043946 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4eb970-352c-4d60-a97a-62d77d67d7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:45.044962 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4eb970-352c-4d60-a97a-62d77d67d7a6 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:45.045963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4eb970-352c-4d60-a97a-62d77d67d7a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:45.046477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4eb970-352c-4d60-a97a-62d77d67d7a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:45.048085 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4eb970-352c-4d60-a97a-62d77d67d7a6 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:45.057925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 173/338] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:45.064016 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a64bd4-f038-4a68-9c8c-f09e54ab5ab6 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:45.163398 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889076e2-f3c2-42b8-b3b0-ff86fa32282e None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:45.164102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889076e2-f3c2-42b8-b3b0-ff86fa32282e None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:45.164602 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889076e2-f3c2-42b8-b3b0-ff86fa32282e None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:45.166422 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 174/339] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:25:45.172678 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70264a8c-2865-4eba-a9ff-b2f92473ec28 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:45.173379 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70264a8c-2865-4eba-a9ff-b2f92473ec28 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:45.174014 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70264a8c-2865-4eba-a9ff-b2f92473ec28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:45.204451 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-70264a8c-2865-4eba-a9ff-b2f92473ec28 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:45.206028 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-70264a8c-2865-4eba-a9ff-b2f92473ec28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:45.234025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a64bd4-f038-4a68-9c8c-f09e54ab5ab6 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:45.318893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adb262-4741-49b1-ab7d-82fae9e994d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:45.319491 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adb262-4741-49b1-ab7d-82fae9e994d0 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:45.319778 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adb262-4741-49b1-ab7d-82fae9e994d0 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:45.320055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adb262-4741-49b1-ab7d-82fae9e994d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:45.328111 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 175/340] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:45.334593 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:45.370609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70264a8c-2865-4eba-a9ff-b2f92473ec28 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:45.445145 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95adb262-4741-49b1-ab7d-82fae9e994d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:45.446633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95adb262-4741-49b1-ab7d-82fae9e994d0 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:45.591621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70264a8c-2865-4eba-a9ff-b2f92473ec28 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:45.638425 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:45.661455 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1ea07-90ce-48c8-b7cc-2b5ef6808371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:45.662386 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1ea07-90ce-48c8-b7cc-2b5ef6808371 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:45.662670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1ea07-90ce-48c8-b7cc-2b5ef6808371 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:45.662946 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1ea07-90ce-48c8-b7cc-2b5ef6808371 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:45.686251 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1ea07-90ce-48c8-b7cc-2b5ef6808371 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:45.694681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-41a1ea07-90ce-48c8-b7cc-2b5ef6808371 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:45.694681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:45.694681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:45.694681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:45.694681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:45.694681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:45.694681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:45.694681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:45.694681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:45.694681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:45.694681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:45.694681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:45.694681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:45.694681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:45.694681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:45.694681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:45.695592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:45.695592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:45.695592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:45.695592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:45.695592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:45.695592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:45.695592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:45.695592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:45.695592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:45.695592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:45.695592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:45.695592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:45.695592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:45.695592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:45.695592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:45.695592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:45.696389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:45.696389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:45.696389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:45.696389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:45.696389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:45.696389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:45.696389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:45.696389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:45.696389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:45.696389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:45.696389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:45.696389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:45.696389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 176/341] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:25:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:25:45.701813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1ea07-90ce-48c8-b7cc-2b5ef6808371 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:45.710781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 166/342] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:45.720435 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adb262-4741-49b1-ab7d-82fae9e994d0 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:45.815985 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:45.875605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1ea07-90ce-48c8-b7cc-2b5ef6808371 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:45.898415 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adb262-4741-49b1-ab7d-82fae9e994d0 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:45.916832 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b32e48-8728-44d5-93cf-7a285c7b0e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:45.917693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b32e48-8728-44d5-93cf-7a285c7b0e65 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:45.917984 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b32e48-8728-44d5-93cf-7a285c7b0e65 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:45.918259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b32e48-8728-44d5-93cf-7a285c7b0e65 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:45.919871 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b32e48-8728-44d5-93cf-7a285c7b0e65 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:45.920183 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b32e48-8728-44d5-93cf-7a285c7b0e65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:45.921619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b32e48-8728-44d5-93cf-7a285c7b0e65 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:45.933292 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 177/343] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:25:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:45.935026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eae752-2d17-42b7-afc5-f499375494b0 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:45.935317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eae752-2d17-42b7-afc5-f499375494b0 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:45.935593 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eae752-2d17-42b7-afc5-f499375494b0 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:45.937041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 178/344] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:45.938273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9421e1-32dc-41e2-abb3-c085c20aca2f None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:45.938558 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9421e1-32dc-41e2-abb3-c085c20aca2f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:45.938833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9421e1-32dc-41e2-abb3-c085c20aca2f None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:45.940382 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 179/345] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:45.941227 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9421e1-32dc-41e2-abb3-c085c20aca2f None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:45.942256 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004c509-1002-4242-88b4-e2b67579c904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:45.943215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004c509-1002-4242-88b4-e2b67579c904 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:45.943507 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004c509-1002-4242-88b4-e2b67579c904 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:45.943778 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004c509-1002-4242-88b4-e2b67579c904 None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:45.946958 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004c509-1002-4242-88b4-e2b67579c904 None admin] RBAC: Authorizing `identity:get_project(project_id=3790dd61247c450ebbf3ef2e90a818e0)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:45.948517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004c509-1002-4242-88b4-e2b67579c904 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:45.951092 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 167/346] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:45 2022] GET /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:45.952331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157a504-84c3-4cf4-8ec2-abecffc15b27 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:45.952620 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157a504-84c3-4cf4-8ec2-abecffc15b27 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:45.952901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157a504-84c3-4cf4-8ec2-abecffc15b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:45.976919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-b157a504-84c3-4cf4-8ec2-abecffc15b27 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:45.978184 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-b157a504-84c3-4cf4-8ec2-abecffc15b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:46.004198 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:46.057369 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 168/347] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:46.060033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8094f-25be-46c9-b0eb-a526002c558a None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:46.060618 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8094f-25be-46c9-b0eb-a526002c558a None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:46.061101 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8094f-25be-46c9-b0eb-a526002c558a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:46.074144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9421e1-32dc-41e2-abb3-c085c20aca2f None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:46.081055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-d5a8094f-25be-46c9-b0eb-a526002c558a None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:46.082574 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d5a8094f-25be-46c9-b0eb-a526002c558a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:46.111621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03098152-0ece-40b8-a997-064156281fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:46.112301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03098152-0ece-40b8-a997-064156281fbf None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:46.112587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03098152-0ece-40b8-a997-064156281fbf None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:46.112864 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03098152-0ece-40b8-a997-064156281fbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:46.139099 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:46.162505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 169/348] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:46.163897 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8094f-25be-46c9-b0eb-a526002c558a None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:46.188614 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03098152-0ece-40b8-a997-064156281fbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:46.190133 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03098152-0ece-40b8-a997-064156281fbf None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:46.238161 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:46.280662 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8094f-25be-46c9-b0eb-a526002c558a None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:46.299273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 180/349] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:46.300157 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03098152-0ece-40b8-a997-064156281fbf None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:46.320741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a312fae-139e-48b5-8fe0-5b5273923f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:46.321986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a312fae-139e-48b5-8fe0-5b5273923f29 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:46.322510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a312fae-139e-48b5-8fe0-5b5273923f29 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:46.322992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a312fae-139e-48b5-8fe0-5b5273923f29 None admin] PATH_INFO: `/v3/groups/8a1fb47776af46f6a4e08488588647d9` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:46.331718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a312fae-139e-48b5-8fe0-5b5273923f29 None admin] RBAC: Authorizing `identity:get_group(group_id=8a1fb47776af46f6a4e08488588647d9)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:46.333696 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a312fae-139e-48b5-8fe0-5b5273923f29 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:46.335773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 170/350] 10.222.0.71 () {56 vars in 1180 bytes} [Thu Jul 21 20:25:46 2022] GET /identity/v3/groups/8a1fb47776af46f6a4e08488588647d9 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:46.336894 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a312fae-139e-48b5-8fe0-5b5273923f29 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:46.379252 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:46.412201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03098152-0ece-40b8-a997-064156281fbf None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:46.451485 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a312fae-139e-48b5-8fe0-5b5273923f29 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:46.456160 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ceaba-423a-4d31-bae5-b27284c7faa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:46.457267 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ceaba-423a-4d31-bae5-b27284c7faa2 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:46.457758 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ceaba-423a-4d31-bae5-b27284c7faa2 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:46.458252 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ceaba-423a-4d31-bae5-b27284c7faa2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:46.471076 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ceaba-423a-4d31-bae5-b27284c7faa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:46.486667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-477ceaba-423a-4d31-bae5-b27284c7faa2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:25:46.486667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:46.486667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:46.486667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:46.486667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:46.486667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:46.486667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:46.486667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:46.486667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:46.486667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:46.486667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:46.486667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:46.486667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:46.486667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:46.486667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:46.486667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:46.487841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:46.487841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:46.487841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:46.487841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:46.487841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:46.487841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:46.487841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:46.487841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:46.487841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:46.487841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:46.487841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:46.487841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:46.487841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:46.487841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:46.487841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:46.487841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:46.488855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:46.488855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:46.488855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:46.488855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:46.488855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:46.488855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:46.488855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:46.488855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:46.488855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:46.488855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:46.488855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:25:46.488855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:46.491810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 181/351] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:46.492736 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ceaba-423a-4d31-bae5-b27284c7faa2 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:46.497361 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128b471-2bf2-4ace-bf65-ece456175351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:46.498184 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128b471-2bf2-4ace-bf65-ece456175351 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:46.499057 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128b471-2bf2-4ace-bf65-ece456175351 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:46.499057 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128b471-2bf2-4ace-bf65-ece456175351 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:46.506972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:46.521782 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128b471-2bf2-4ace-bf65-ece456175351 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:46.532644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-9128b471-2bf2-4ace-bf65-ece456175351 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:46.532644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:46.532644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:46.532644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:46.532644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:46.532644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:46.532644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:46.532644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:46.532644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:46.532644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:46.532644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:46.532644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:46.532644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:46.532644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:46.532644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:46.532644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:46.533643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:46.533643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:46.533643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:46.533643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:46.533643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:46.533643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:46.533643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:46.533643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:46.533643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:46.533643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:46.533643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:46.533643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:46.533643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:46.533643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:46.533643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:46.533643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:46.534449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:46.534449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:46.534449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:46.534449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:46.534449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:46.534449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:46.534449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:46.534449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:46.534449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:46.534449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:46.534449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:46.534449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:46.535066 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 171/352] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:46.535615 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128b471-2bf2-4ace-bf65-ece456175351 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:46.636750 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ceaba-423a-4d31-bae5-b27284c7faa2 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:46.662758 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:46.675206 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128b471-2bf2-4ace-bf65-ece456175351 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:46.681020 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1422ed-c45b-4cc5-b3c9-f2271ffd9e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:46.681921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1422ed-c45b-4cc5-b3c9-f2271ffd9e40 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:46.682205 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1422ed-c45b-4cc5-b3c9-f2271ffd9e40 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:46.682482 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1422ed-c45b-4cc5-b3c9-f2271ffd9e40 None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:46.683705 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1422ed-c45b-4cc5-b3c9-f2271ffd9e40 None admin] RBAC: Authorizing `identity:get_project(project_id=3790dd61247c450ebbf3ef2e90a818e0)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:46.685794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1422ed-c45b-4cc5-b3c9-f2271ffd9e40 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:46.687821 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 182/353] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:46 2022] GET /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:46.688605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1422ed-c45b-4cc5-b3c9-f2271ffd9e40 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:46.726690 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45057aad-01a8-43bc-94f9-fe829c0bda00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:46.727524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45057aad-01a8-43bc-94f9-fe829c0bda00 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:46.727911 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45057aad-01a8-43bc-94f9-fe829c0bda00 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:46.728289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45057aad-01a8-43bc-94f9-fe829c0bda00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:46.729283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45057aad-01a8-43bc-94f9-fe829c0bda00 None admin] RBAC: Adding query filter params (group.id=8a1fb47776af46f6a4e08488588647d9, role.id=8a0eeb2a21dc4897a58fb5851d58678c, scope.project.id=3790dd61247c450ebbf3ef2e90a818e0) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:46.729705 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45057aad-01a8-43bc-94f9-fe829c0bda00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:46.731278 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45057aad-01a8-43bc-94f9-fe829c0bda00 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:46.739224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 172/354] 10.222.0.71 () {56 vars in 1376 bytes} [Thu Jul 21 20:25:46 2022] GET /identity/v3/role_assignments?group.id=8a1fb47776af46f6a4e08488588647d9&scope.project.id=3790dd61247c450ebbf3ef2e90a818e0&role.id=8a0eeb2a21dc4897a58fb5851d58678c => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:46.740352 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653156bb-3d9d-4a89-986f-a491308b4d92 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:46.740643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653156bb-3d9d-4a89-986f-a491308b4d92 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:46.740925 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653156bb-3d9d-4a89-986f-a491308b4d92 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:46.742213 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 173/355] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:46.742912 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653156bb-3d9d-4a89-986f-a491308b4d92 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:46.778274 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:46.816369 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1422ed-c45b-4cc5-b3c9-f2271ffd9e40 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:46.855716 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182ed25-2e9a-4958-b35d-ccf50a1f8036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:46.857353 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182ed25-2e9a-4958-b35d-ccf50a1f8036 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:46.857866 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182ed25-2e9a-4958-b35d-ccf50a1f8036 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:46.858368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182ed25-2e9a-4958-b35d-ccf50a1f8036 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:46.859358 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182ed25-2e9a-4958-b35d-ccf50a1f8036 None admin] RBAC: Adding query filter params (name=reader) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:46.860013 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182ed25-2e9a-4958-b35d-ccf50a1f8036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:46.861666 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182ed25-2e9a-4958-b35d-ccf50a1f8036 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:46.871674 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 183/356] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:46.873130 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182ed25-2e9a-4958-b35d-ccf50a1f8036 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:46.879908 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653156bb-3d9d-4a89-986f-a491308b4d92 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:46.918925 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9bc82-fc44-4eae-a698-c8e741bfaf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:46.919558 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9bc82-fc44-4eae-a698-c8e741bfaf60 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:46.919845 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9bc82-fc44-4eae-a698-c8e741bfaf60 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:46.920127 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9bc82-fc44-4eae-a698-c8e741bfaf60 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:46.921107 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9bc82-fc44-4eae-a698-c8e741bfaf60 None admin] RBAC: Adding query filter params (name=member) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:46.921673 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9bc82-fc44-4eae-a698-c8e741bfaf60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:46.923330 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9bc82-fc44-4eae-a698-c8e741bfaf60 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:46.934360 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 174/357] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:46.936533 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f990c27-2403-454c-b54b-c66e2f85ac32 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:46.937029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f990c27-2403-454c-b54b-c66e2f85ac32 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:46.937552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f990c27-2403-454c-b54b-c66e2f85ac32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:46.958159 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-6f990c27-2403-454c-b54b-c66e2f85ac32 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:46.959723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-6f990c27-2403-454c-b54b-c66e2f85ac32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:46.971998 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:47.003658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182ed25-2e9a-4958-b35d-ccf50a1f8036 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:47.042280 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 175/358] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:47.043104 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f990c27-2403-454c-b54b-c66e2f85ac32 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:47.060383 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f2eeb-bd46-42c5-873f-51720feff1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:47.062278 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f2eeb-bd46-42c5-873f-51720feff1a6 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:47.064413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f2eeb-bd46-42c5-873f-51720feff1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:47.064413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f2eeb-bd46-42c5-873f-51720feff1a6 None admin] PATH_INFO: `/v3/groups/8a1fb47776af46f6a4e08488588647d9` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:47.065058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f2eeb-bd46-42c5-873f-51720feff1a6 None admin] RBAC: Authorizing `identity:get_group(group_id=8a1fb47776af46f6a4e08488588647d9)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:47.066580 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f2eeb-bd46-42c5-873f-51720feff1a6 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:47.069601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 184/359] 10.222.0.71 () {56 vars in 1180 bytes} [Thu Jul 21 20:25:46 2022] GET /identity/v3/groups/8a1fb47776af46f6a4e08488588647d9 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:47.069601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f2eeb-bd46-42c5-873f-51720feff1a6 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:47.083278 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:47.194213 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f990c27-2403-454c-b54b-c66e2f85ac32 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:47.225725 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f2eeb-bd46-42c5-873f-51720feff1a6 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:47.243679 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20786ab6-1c7b-4f91-af84-a23352be4082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:47.244564 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20786ab6-1c7b-4f91-af84-a23352be4082 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:47.244844 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20786ab6-1c7b-4f91-af84-a23352be4082 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:47.245154 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20786ab6-1c7b-4f91-af84-a23352be4082 None admin] PATH_INFO: `/v3/users/514cebea3b6f46c5840a10f95b2c3a9c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:47.246342 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20786ab6-1c7b-4f91-af84-a23352be4082 None admin] RBAC: Authorizing `identity:get_user(user_id=514cebea3b6f46c5840a10f95b2c3a9c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:47.248427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20786ab6-1c7b-4f91-af84-a23352be4082 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:47.250541 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 176/360] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:25:47 2022] GET /identity/v3/users/514cebea3b6f46c5840a10f95b2c3a9c => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:47.250878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20786ab6-1c7b-4f91-af84-a23352be4082 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:47.274758 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:47.278917 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac334a9-cdd5-4f1b-a27e-dc1f34df4729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:47.279577 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac334a9-cdd5-4f1b-a27e-dc1f34df4729 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:47.279862 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac334a9-cdd5-4f1b-a27e-dc1f34df4729 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:47.280142 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac334a9-cdd5-4f1b-a27e-dc1f34df4729 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:47.376349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac334a9-cdd5-4f1b-a27e-dc1f34df4729 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:47.378064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac334a9-cdd5-4f1b-a27e-dc1f34df4729 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:47.381951 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:47.391782 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20786ab6-1c7b-4f91-af84-a23352be4082 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:47.435493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c89e56-c202-4b8a-8cdd-29b7071c4f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:47.436405 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c89e56-c202-4b8a-8cdd-29b7071c4f5e None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:47.436860 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c89e56-c202-4b8a-8cdd-29b7071c4f5e None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:47.437311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c89e56-c202-4b8a-8cdd-29b7071c4f5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:47.438343 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c89e56-c202-4b8a-8cdd-29b7071c4f5e None admin] RBAC: Adding query filter params (group.id=8a1fb47776af46f6a4e08488588647d9, role.id=bad7164dd96e4048bab9360ef3f6769e, scope.project.id=3790dd61247c450ebbf3ef2e90a818e0) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:47.438662 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c89e56-c202-4b8a-8cdd-29b7071c4f5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:47.440106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c89e56-c202-4b8a-8cdd-29b7071c4f5e None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:47.448729 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 177/361] 10.222.0.71 () {56 vars in 1376 bytes} [Thu Jul 21 20:25:47 2022] GET /identity/v3/role_assignments?group.id=8a1fb47776af46f6a4e08488588647d9&scope.project.id=3790dd61247c450ebbf3ef2e90a818e0&role.id=bad7164dd96e4048bab9360ef3f6769e => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:47.453991 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c89e56-c202-4b8a-8cdd-29b7071c4f5e None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:47.511603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 185/362] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:47.512843 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880876b-109e-4937-bf70-f5fb8136d3ac None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:47.517413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880876b-109e-4937-bf70-f5fb8136d3ac None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:47.517728 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880876b-109e-4937-bf70-f5fb8136d3ac None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:47.519223 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 186/363] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:47.519979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880876b-109e-4937-bf70-f5fb8136d3ac None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:47.555334 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:47.583058 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c89e56-c202-4b8a-8cdd-29b7071c4f5e None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:47.625388 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb84da2-4c94-4a76-be7a-1e1f6cbaae90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:47.626683 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb84da2-4c94-4a76-be7a-1e1f6cbaae90 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:47.627266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb84da2-4c94-4a76-be7a-1e1f6cbaae90 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:47.627741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb84da2-4c94-4a76-be7a-1e1f6cbaae90 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:47.640156 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb84da2-4c94-4a76-be7a-1e1f6cbaae90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:47.645543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880876b-109e-4937-bf70-f5fb8136d3ac None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:47.647981 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:47.651021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-afb84da2-4c94-4a76-be7a-1e1f6cbaae90 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:47.651021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:47.651021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:47.651021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:47.651021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:47.651021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:47.651021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:47.651021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:47.651021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:47.651021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:47.651021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:47.651021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:47.651021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:47.651021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:47.651021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:47.651021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:47.652048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:47.652048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:47.652048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:47.652048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:47.652048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:47.652048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:47.652048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:47.652048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:47.652048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:47.652048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:47.652048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:47.652048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:47.652048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:47.652048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:47.652048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:47.652048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:47.652855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:47.652855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:47.652855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:47.652855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:47.652855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:47.652855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:47.652855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:47.652855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:47.652855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:47.652855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:47.652855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:47.652855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:47.655877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 178/364] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:25:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:47.656896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb84da2-4c94-4a76-be7a-1e1f6cbaae90 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:47.683438 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4e8b3-2638-4b35-aa3d-f03c792d3a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:47.684158 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4e8b3-2638-4b35-aa3d-f03c792d3a0c None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:47.684366 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4e8b3-2638-4b35-aa3d-f03c792d3a0c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:47.684637 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4e8b3-2638-4b35-aa3d-f03c792d3a0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:47.685623 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e4e8b3-2638-4b35-aa3d-f03c792d3a0c None admin] RBAC: Adding query filter params (role.id=bc82ed55215d47b791d5aa25699e7e2b, user.id=514cebea3b6f46c5840a10f95b2c3a9c, scope.system=all) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:47.685932 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e4e8b3-2638-4b35-aa3d-f03c792d3a0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:47.687446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e4e8b3-2638-4b35-aa3d-f03c792d3a0c None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:47.705631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 187/365] 10.222.0.71 () {56 vars in 1308 bytes} [Thu Jul 21 20:25:47 2022] GET /identity/v3/role_assignments?user.id=514cebea3b6f46c5840a10f95b2c3a9c&scope.system=all&role.id=bc82ed55215d47b791d5aa25699e7e2b => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:47.707035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc25d5d-b301-4844-a8d6-2c07bee45d99 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:47.707320 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc25d5d-b301-4844-a8d6-2c07bee45d99 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:47.707595 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc25d5d-b301-4844-a8d6-2c07bee45d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:47.739933 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-5bc25d5d-b301-4844-a8d6-2c07bee45d99 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:47.741181 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-5bc25d5d-b301-4844-a8d6-2c07bee45d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:47.780575 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb84da2-4c94-4a76-be7a-1e1f6cbaae90 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:47.820914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcd2c9-f97d-4c92-bca0-b3584162a58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:47.822207 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcd2c9-f97d-4c92-bca0-b3584162a58f None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:47.822693 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcd2c9-f97d-4c92-bca0-b3584162a58f None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:47.823169 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcd2c9-f97d-4c92-bca0-b3584162a58f None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:47.824593 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dcd2c9-f97d-4c92-bca0-b3584162a58f None admin] RBAC: Authorizing `identity:get_project(project_id=956f270e083e4b15b70833346dcb8e19)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:47.826369 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dcd2c9-f97d-4c92-bca0-b3584162a58f None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:47.828327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 179/366] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:47 2022] GET /identity/v3/projects/956f270e083e4b15b70833346dcb8e19 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:47.829322 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcd2c9-f97d-4c92-bca0-b3584162a58f None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:47.839017 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:47.848308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 188/367] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:47.850695 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc25d5d-b301-4844-a8d6-2c07bee45d99 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:47.995318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcd2c9-f97d-4c92-bca0-b3584162a58f None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:47.996196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:48.010295 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc25d5d-b301-4844-a8d6-2c07bee45d99 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:48.046715 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff9469-f91d-4bf3-8ff5-1a8655736dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:48.048081 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff9469-f91d-4bf3-8ff5-1a8655736dfe None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:48.048828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff9469-f91d-4bf3-8ff5-1a8655736dfe None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:48.049764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff9469-f91d-4bf3-8ff5-1a8655736dfe None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:48.050924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff9469-f91d-4bf3-8ff5-1a8655736dfe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:48.051574 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff9469-f91d-4bf3-8ff5-1a8655736dfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:48.053594 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff9469-f91d-4bf3-8ff5-1a8655736dfe None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:48.055380 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500b385-541a-402c-a46d-1c20d4b70b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:48.056350 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500b385-541a-402c-a46d-1c20d4b70b16 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:48.056698 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500b385-541a-402c-a46d-1c20d4b70b16 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:48.057044 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500b385-541a-402c-a46d-1c20d4b70b16 None admin] PATH_INFO: `/v3/groups/8a1fb47776af46f6a4e08488588647d9` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:48.058613 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500b385-541a-402c-a46d-1c20d4b70b16 None admin] RBAC: Authorizing `identity:get_group(group_id=8a1fb47776af46f6a4e08488588647d9)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:48.060232 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500b385-541a-402c-a46d-1c20d4b70b16 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:48.063428 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 189/368] 10.222.0.71 () {56 vars in 1180 bytes} [Thu Jul 21 20:25:47 2022] GET /identity/v3/groups/8a1fb47776af46f6a4e08488588647d9 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:48.064520 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500b385-541a-402c-a46d-1c20d4b70b16 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:48.069516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 180/369] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:25:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:48.077294 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff9469-f91d-4bf3-8ff5-1a8655736dfe None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:48.173815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:48.207080 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500b385-541a-402c-a46d-1c20d4b70b16 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:48.230258 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff9469-f91d-4bf3-8ff5-1a8655736dfe None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:48.252411 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ea4d7-3d4b-4a99-86ee-c62fe4269833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:48.253162 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ea4d7-3d4b-4a99-86ee-c62fe4269833 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:48.253441 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ea4d7-3d4b-4a99-86ee-c62fe4269833 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:48.253711 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ea4d7-3d4b-4a99-86ee-c62fe4269833 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:48.266808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ea4d7-3d4b-4a99-86ee-c62fe4269833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:48.274398 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53c36c-52f6-4f6e-8788-590b4037a4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:48.275064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-6f0ea4d7-3d4b-4a99-86ee-c62fe4269833 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:48.275064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:48.275064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:48.275064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:48.275064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:48.275064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:48.275064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:48.275064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:48.275064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:48.275064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:48.275064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:48.275064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:48.275064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:48.275064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:48.275064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:48.275064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:48.275971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:48.275971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:48.275971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:48.275971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:48.275971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:48.275971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:48.275971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:48.275971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:48.275971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:48.275971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:48.275971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:48.275971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:48.275971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:48.275971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:48.275971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:48.275971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:48.276796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:48.276796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:48.276796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:48.276796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:48.276796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:48.276796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:48.276796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:48.276796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:48.276796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:48.276796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:48.276796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:48.276796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:48.276796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 190/370] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:25:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:48.277523 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ea4d7-3d4b-4a99-86ee-c62fe4269833 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:48.278536 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53c36c-52f6-4f6e-8788-590b4037a4b1 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:48.279033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53c36c-52f6-4f6e-8788-590b4037a4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:48.279513 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53c36c-52f6-4f6e-8788-590b4037a4b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:48.307915 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:48.369751 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53c36c-52f6-4f6e-8788-590b4037a4b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:48.370701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53c36c-52f6-4f6e-8788-590b4037a4b1 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:48.410247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ea4d7-3d4b-4a99-86ee-c62fe4269833 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:48.453595 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f42306-cb7b-44e5-8d77-c6774e842656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:48.454868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f42306-cb7b-44e5-8d77-c6774e842656 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:48.455907 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f42306-cb7b-44e5-8d77-c6774e842656 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:48.456548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f42306-cb7b-44e5-8d77-c6774e842656 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:48.457698 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f42306-cb7b-44e5-8d77-c6774e842656 None admin] RBAC: Adding query filter params (group.id=8a1fb47776af46f6a4e08488588647d9, role.id=8a0eeb2a21dc4897a58fb5851d58678c, scope.project.id=956f270e083e4b15b70833346dcb8e19) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:48.458166 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f42306-cb7b-44e5-8d77-c6774e842656 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:48.460656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f42306-cb7b-44e5-8d77-c6774e842656 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:48.469272 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 191/371] 10.222.0.71 () {56 vars in 1376 bytes} [Thu Jul 21 20:25:48 2022] GET /identity/v3/role_assignments?group.id=8a1fb47776af46f6a4e08488588647d9&scope.project.id=956f270e083e4b15b70833346dcb8e19&role.id=8a0eeb2a21dc4897a58fb5851d58678c => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:48.469762 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:48.470120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f42306-cb7b-44e5-8d77-c6774e842656 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:48.544900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 181/372] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:48.550823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53c36c-52f6-4f6e-8788-590b4037a4b1 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:48.629367 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:48.651938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f42306-cb7b-44e5-8d77-c6774e842656 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:48.702823 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85514eb7-58b1-4465-a8ff-d7f8d3851d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:48.703515 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85514eb7-58b1-4465-a8ff-d7f8d3851d1c None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:48.703792 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85514eb7-58b1-4465-a8ff-d7f8d3851d1c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:48.704064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85514eb7-58b1-4465-a8ff-d7f8d3851d1c None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:48.704832 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85514eb7-58b1-4465-a8ff-d7f8d3851d1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:48.705200 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85514eb7-58b1-4465-a8ff-d7f8d3851d1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:48.706601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85514eb7-58b1-4465-a8ff-d7f8d3851d1c None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:48.720694 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 192/373] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:48.726517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85514eb7-58b1-4465-a8ff-d7f8d3851d1c None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:48.828229 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53c36c-52f6-4f6e-8788-590b4037a4b1 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:48.857910 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85514eb7-58b1-4465-a8ff-d7f8d3851d1c None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:48.905900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252c435-aae4-4e5b-86f3-5a86a1ae0cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:48.906771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252c435-aae4-4e5b-86f3-5a86a1ae0cf4 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:48.907051 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252c435-aae4-4e5b-86f3-5a86a1ae0cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:48.907324 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252c435-aae4-4e5b-86f3-5a86a1ae0cf4 None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:48.909596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252c435-aae4-4e5b-86f3-5a86a1ae0cf4 None admin] RBAC: Authorizing `identity:get_project(project_id=956f270e083e4b15b70833346dcb8e19)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:48.911044 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252c435-aae4-4e5b-86f3-5a86a1ae0cf4 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:48.912950 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 182/374] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:48 2022] GET /identity/v3/projects/956f270e083e4b15b70833346dcb8e19 => generated 302 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:48.918544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252c435-aae4-4e5b-86f3-5a86a1ae0cf4 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:48.930555 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f61a3f-1b92-41e6-aefc-90f7207494b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:48.931219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f61a3f-1b92-41e6-aefc-90f7207494b4 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:48.931502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f61a3f-1b92-41e6-aefc-90f7207494b4 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:48.931781 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f61a3f-1b92-41e6-aefc-90f7207494b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:48.963410 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:48.988885 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:49.063600 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252c435-aae4-4e5b-86f3-5a86a1ae0cf4 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:49.105516 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f61a3f-1b92-41e6-aefc-90f7207494b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:49.106545 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f52274-f9d1-4e6d-8ebc-c956d07c625b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:49.106990 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f61a3f-1b92-41e6-aefc-90f7207494b4 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:49.107494 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f52274-f9d1-4e6d-8ebc-c956d07c625b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:49.107778 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f52274-f9d1-4e6d-8ebc-c956d07c625b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:49.108052 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f52274-f9d1-4e6d-8ebc-c956d07c625b None admin] PATH_INFO: `/v3/groups/8a1fb47776af46f6a4e08488588647d9` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:49.109435 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f52274-f9d1-4e6d-8ebc-c956d07c625b None admin] RBAC: Authorizing `identity:get_group(group_id=8a1fb47776af46f6a4e08488588647d9)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:49.110991 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f52274-f9d1-4e6d-8ebc-c956d07c625b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:49.113880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 183/375] 10.222.0.71 () {56 vars in 1180 bytes} [Thu Jul 21 20:25:48 2022] GET /identity/v3/groups/8a1fb47776af46f6a4e08488588647d9 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:49.119643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f52274-f9d1-4e6d-8ebc-c956d07c625b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:49.250543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 193/376] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 20:25:49.251878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a2686-aa82-40f6-a511-30c6be5b8716 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:49.252166 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a2686-aa82-40f6-a511-30c6be5b8716 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:49.252444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a2686-aa82-40f6-a511-30c6be5b8716 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:49.253917 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 194/377] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:49.259304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f206ea-f46c-4d6e-824e-0eb7006fc2fb None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:49.259588 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f206ea-f46c-4d6e-824e-0eb7006fc2fb None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:49.259870 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f206ea-f46c-4d6e-824e-0eb7006fc2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:49.269723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f52274-f9d1-4e6d-8ebc-c956d07c625b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:49.272685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:49.279392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-73f206ea-f46c-4d6e-824e-0eb7006fc2fb None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:49.281011 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-73f206ea-f46c-4d6e-824e-0eb7006fc2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:49.317053 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b6af6-96e7-4fc6-8ee5-7d3b08993212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:49.317782 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b6af6-96e7-4fc6-8ee5-7d3b08993212 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:49.318059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b6af6-96e7-4fc6-8ee5-7d3b08993212 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:49.318333 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b6af6-96e7-4fc6-8ee5-7d3b08993212 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:49.319174 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b6af6-96e7-4fc6-8ee5-7d3b08993212 None admin] RBAC: Adding query filter params (group.id=8a1fb47776af46f6a4e08488588647d9, role.id=bad7164dd96e4048bab9360ef3f6769e, scope.project.id=956f270e083e4b15b70833346dcb8e19) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:49.319489 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b6af6-96e7-4fc6-8ee5-7d3b08993212 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:49.321010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b6af6-96e7-4fc6-8ee5-7d3b08993212 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:49.331850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 184/378] 10.222.0.71 () {56 vars in 1376 bytes} [Thu Jul 21 20:25:49 2022] GET /identity/v3/role_assignments?group.id=8a1fb47776af46f6a4e08488588647d9&scope.project.id=956f270e083e4b15b70833346dcb8e19&role.id=bad7164dd96e4048bab9360ef3f6769e => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:49.333134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b6af6-96e7-4fc6-8ee5-7d3b08993212 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:49.342803 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:49.379845 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 195/379] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:49.432829 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f206ea-f46c-4d6e-824e-0eb7006fc2fb None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:49.485122 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b6af6-96e7-4fc6-8ee5-7d3b08993212 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:49.524066 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09c3d6-a80c-4522-a4b8-adca18409c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:49.525098 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09c3d6-a80c-4522-a4b8-adca18409c1d None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:49.525304 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09c3d6-a80c-4522-a4b8-adca18409c1d None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:49.525539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09c3d6-a80c-4522-a4b8-adca18409c1d None admin] PATH_INFO: `/v3/projects/3dd46499ce284e948f969ddbeb31196c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:49.526762 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09c3d6-a80c-4522-a4b8-adca18409c1d None admin] RBAC: Authorizing `identity:get_project(project_id=3dd46499ce284e948f969ddbeb31196c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:49.528243 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09c3d6-a80c-4522-a4b8-adca18409c1d None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:49.531374 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 185/380] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:49 2022] GET /identity/v3/projects/3dd46499ce284e948f969ddbeb31196c => generated 344 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:49.536563 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09c3d6-a80c-4522-a4b8-adca18409c1d None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:49.577853 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:49.664371 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09c3d6-a80c-4522-a4b8-adca18409c1d None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:49.679077 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f206ea-f46c-4d6e-824e-0eb7006fc2fb None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:49.707013 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18315654-ca51-435e-8861-9ba136afe044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:49.708474 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18315654-ca51-435e-8861-9ba136afe044 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:49.708996 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18315654-ca51-435e-8861-9ba136afe044 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:49.709524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18315654-ca51-435e-8861-9ba136afe044 None admin] PATH_INFO: `/v3/groups/1151069a2c68436ebefc14ce92ef935f` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:49.716036 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18315654-ca51-435e-8861-9ba136afe044 None admin] RBAC: Authorizing `identity:get_group(group_id=1151069a2c68436ebefc14ce92ef935f)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:49.717999 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18315654-ca51-435e-8861-9ba136afe044 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:49.721510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 186/381] 10.222.0.71 () {56 vars in 1180 bytes} [Thu Jul 21 20:25:49 2022] GET /identity/v3/groups/1151069a2c68436ebefc14ce92ef935f => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:49.728369 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18315654-ca51-435e-8861-9ba136afe044 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:49.768421 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e533b-c3e5-49c0-946c-49b36353ed8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:49.770452 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e533b-c3e5-49c0-946c-49b36353ed8b None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:49.774148 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e533b-c3e5-49c0-946c-49b36353ed8b None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:49.774730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e533b-c3e5-49c0-946c-49b36353ed8b None admin] PATH_INFO: `/v3/roles/member` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:49.799132 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e533b-c3e5-49c0-946c-49b36353ed8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:49.816034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-c78e533b-c3e5-49c0-946c-49b36353ed8b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:49.816034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:49.816034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:49.816034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:49.816034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:49.816034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:49.816034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:49.816034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:49.816034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:49.816034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:49.816034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:49.816034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:49.816034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:49.816034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:49.816034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:49.816034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:49.817025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:49.817025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:49.817025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:49.817025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:49.817025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:49.817025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:49.817025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:49.817025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:49.817025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:49.817025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:49.817025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:49.817025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:49.817025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:49.817025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:49.817025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:49.817025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:49.817881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:49.817881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:49.817881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:49.817881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:49.817881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:49.817881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:49.817881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:49.817881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:49.817881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:49.817881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:49.817881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:49.817881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:49.825060 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 196/382] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:25:49.829880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e533b-c3e5-49c0-946c-49b36353ed8b None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:49.885581 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:49.930066 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:49.934520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18315654-ca51-435e-8861-9ba136afe044 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:49.983535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e533b-c3e5-49c0-946c-49b36353ed8b None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:50.025808 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8833d1-49db-45b1-a757-67a91409b65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:50.026482 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8833d1-49db-45b1-a757-67a91409b65c None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:50.026769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8833d1-49db-45b1-a757-67a91409b65c None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:50.027041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8833d1-49db-45b1-a757-67a91409b65c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:50.027898 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8833d1-49db-45b1-a757-67a91409b65c None admin] RBAC: Adding query filter params (group.id=1151069a2c68436ebefc14ce92ef935f, role.id=19f5e72c0be44a1197c7288de82e79e8, scope.project.id=3dd46499ce284e948f969ddbeb31196c) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:50.028211 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8833d1-49db-45b1-a757-67a91409b65c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:50.030263 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8833d1-49db-45b1-a757-67a91409b65c None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:50.037054 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629c67a-544a-41c5-b7d7-21ece62d492d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:50.037745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629c67a-544a-41c5-b7d7-21ece62d492d None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:50.038028 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629c67a-544a-41c5-b7d7-21ece62d492d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:50.038752 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629c67a-544a-41c5-b7d7-21ece62d492d None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:50.040018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629c67a-544a-41c5-b7d7-21ece62d492d None admin] RBAC: Adding query filter params (name=member) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:50.040334 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629c67a-544a-41c5-b7d7-21ece62d492d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:50.041877 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629c67a-544a-41c5-b7d7-21ece62d492d None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:50.041988 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 187/383] 10.222.0.71 () {56 vars in 1376 bytes} [Thu Jul 21 20:25:49 2022] GET /identity/v3/role_assignments?group.id=1151069a2c68436ebefc14ce92ef935f&scope.project.id=3dd46499ce284e948f969ddbeb31196c&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 260 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:50.056625 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 197/384] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:50.063132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8833d1-49db-45b1-a757-67a91409b65c None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:50.219911 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8833d1-49db-45b1-a757-67a91409b65c None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:50.267160 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e8e46-95dd-41b6-8096-ba03240b5347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:50.268032 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e8e46-95dd-41b6-8096-ba03240b5347 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:50.268325 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e8e46-95dd-41b6-8096-ba03240b5347 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:50.268607 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e8e46-95dd-41b6-8096-ba03240b5347 None admin] PATH_INFO: `/v3/groups/8a1fb47776af46f6a4e08488588647d9` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:50.269872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203ccf5-5b2a-4c10-a3ac-fea77e39bf24 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:50.269999 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e8e46-95dd-41b6-8096-ba03240b5347 None admin] RBAC: Authorizing `identity:get_group(group_id=8a1fb47776af46f6a4e08488588647d9)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:50.270168 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203ccf5-5b2a-4c10-a3ac-fea77e39bf24 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:50.270435 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203ccf5-5b2a-4c10-a3ac-fea77e39bf24 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:50.271318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e8e46-95dd-41b6-8096-ba03240b5347 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:50.272077 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 198/385] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:50.273263 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 188/386] 10.222.0.71 () {56 vars in 1180 bytes} [Thu Jul 21 20:25:50 2022] GET /identity/v3/groups/8a1fb47776af46f6a4e08488588647d9 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:50.279081 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203ccf5-5b2a-4c10-a3ac-fea77e39bf24 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:50.281144 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a467d-135f-4b14-865f-333bf70508fa None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:50.281440 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a467d-135f-4b14-865f-333bf70508fa None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:50.281722 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a467d-135f-4b14-865f-333bf70508fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:50.307157 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-482a467d-135f-4b14-865f-333bf70508fa None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:50.308384 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-482a467d-135f-4b14-865f-333bf70508fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:50.369534 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:50.451757 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 189/387] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:50.476475 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:50.476640 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a467d-135f-4b14-865f-333bf70508fa None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:50.492873 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203ccf5-5b2a-4c10-a3ac-fea77e39bf24 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:50.552943 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821ed13-3d4b-4ead-acc3-9f5ff8763251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:50.553967 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821ed13-3d4b-4ead-acc3-9f5ff8763251 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:50.554250 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821ed13-3d4b-4ead-acc3-9f5ff8763251 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:50.554524 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821ed13-3d4b-4ead-acc3-9f5ff8763251 None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:50.555760 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821ed13-3d4b-4ead-acc3-9f5ff8763251 None admin] RBAC: Authorizing `identity:get_project(project_id=3790dd61247c450ebbf3ef2e90a818e0)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:50.557311 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821ed13-3d4b-4ead-acc3-9f5ff8763251 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:50.560059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 199/388] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:50 2022] GET /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0 => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:50.566562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821ed13-3d4b-4ead-acc3-9f5ff8763251 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:50.695855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a467d-135f-4b14-865f-333bf70508fa None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:50.708840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821ed13-3d4b-4ead-acc3-9f5ff8763251 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:50.763232 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea14c6-443f-45fc-b763-b14b34e1c85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:50.763993 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea14c6-443f-45fc-b763-b14b34e1c85a None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:50.764275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea14c6-443f-45fc-b763-b14b34e1c85a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:50.764549 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea14c6-443f-45fc-b763-b14b34e1c85a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:50.768845 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d42b2b-df7e-42c1-ba6a-d9c1bddb868d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:50.769762 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d42b2b-df7e-42c1-ba6a-d9c1bddb868d None admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:50.770042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d42b2b-df7e-42c1-ba6a-d9c1bddb868d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:50.770317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d42b2b-df7e-42c1-ba6a-d9c1bddb868d None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0/groups/8a1fb47776af46f6a4e08488588647d9/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:50.773384 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d42b2b-df7e-42c1-ba6a-d9c1bddb868d None admin] RBAC: Authorizing `identity:create_grant(project_id=3790dd61247c450ebbf3ef2e90a818e0, group_id=8a1fb47776af46f6a4e08488588647d9, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:50.774885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d42b2b-df7e-42c1-ba6a-d9c1bddb868d None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:50.789041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea14c6-443f-45fc-b763-b14b34e1c85a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:50.800846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-95ea14c6-443f-45fc-b763-b14b34e1c85a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:50.800846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:50.800846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:50.800846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:50.800846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:50.800846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:50.800846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:50.800846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:50.800846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:50.800846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:50.800846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:50.800846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:50.800846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:50.800846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:50.800846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:50.800846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:50.801863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:50.801863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:50.801863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:50.801863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:50.801863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:50.801863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:50.801863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:50.801863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:50.801863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:50.801863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:50.801863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:50.801863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:50.801863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:50.801863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:50.801863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:50.801863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:50.802775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:50.802775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:50.802775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:50.802775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:50.802775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:50.802775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:50.802775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:50.802775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:50.802775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:50.802775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:50.802775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:50.802775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:50.802775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 190/389] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:25:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:50.804158 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 200/390] 10.222.0.71 () {58 vars in 1442 bytes} [Thu Jul 21 20:25:50 2022] PUT /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0/groups/8a1fb47776af46f6a4e08488588647d9/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:25:50.810341 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea14c6-443f-45fc-b763-b14b34e1c85a None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:50.850472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:50.962190 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea14c6-443f-45fc-b763-b14b34e1c85a None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:51.018120 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d939891-18ec-47dd-bcad-4faef6cbb81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:51.018797 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d939891-18ec-47dd-bcad-4faef6cbb81f None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:51.019087 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d939891-18ec-47dd-bcad-4faef6cbb81f None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:51.019364 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d939891-18ec-47dd-bcad-4faef6cbb81f None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:51.020159 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d939891-18ec-47dd-bcad-4faef6cbb81f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:51.020464 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d939891-18ec-47dd-bcad-4faef6cbb81f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:51.021991 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d939891-18ec-47dd-bcad-4faef6cbb81f None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:51.032498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 191/391] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:25:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:51.038185 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d42b2b-df7e-42c1-ba6a-d9c1bddb868d None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:51.052011 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebeb91-9912-4e5f-99a8-983085fcd9d4 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:51.052326 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebeb91-9912-4e5f-99a8-983085fcd9d4 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:51.052564 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebeb91-9912-4e5f-99a8-983085fcd9d4 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:51.054314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 192/392] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:25:51.066852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0441f-1019-4154-bc70-e243290203f9 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:51.067155 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0441f-1019-4154-bc70-e243290203f9 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:51.067412 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0441f-1019-4154-bc70-e243290203f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:51.092166 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-b8f0441f-1019-4154-bc70-e243290203f9 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:51.093449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-b8f0441f-1019-4154-bc70-e243290203f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:51.107487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:51.195272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 193/393] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:51.219733 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f0441f-1019-4154-bc70-e243290203f9 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:51.225430 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d42b2b-df7e-42c1-ba6a-d9c1bddb868d None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:51.245579 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:51.287024 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b49c0f-1407-4b12-bd34-1ff348361a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:51.287814 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b49c0f-1407-4b12-bd34-1ff348361a5e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:51.288099 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b49c0f-1407-4b12-bd34-1ff348361a5e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:51.288378 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b49c0f-1407-4b12-bd34-1ff348361a5e None admin] PATH_INFO: `/v3/groups/8a1fb47776af46f6a4e08488588647d9` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:51.293791 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b49c0f-1407-4b12-bd34-1ff348361a5e None admin] RBAC: Authorizing `identity:get_group(group_id=8a1fb47776af46f6a4e08488588647d9)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:51.295820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b49c0f-1407-4b12-bd34-1ff348361a5e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:51.297838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 201/394] 10.222.0.71 () {56 vars in 1180 bytes} [Thu Jul 21 20:25:51 2022] GET /identity/v3/groups/8a1fb47776af46f6a4e08488588647d9 => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:51.305983 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b49c0f-1407-4b12-bd34-1ff348361a5e None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:51.334088 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f0441f-1019-4154-bc70-e243290203f9 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:51.372952 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475dbd8-91ab-49f1-aeb5-822d3b13822a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:51.373882 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475dbd8-91ab-49f1-aeb5-822d3b13822a None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:51.374164 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475dbd8-91ab-49f1-aeb5-822d3b13822a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:51.374435 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475dbd8-91ab-49f1-aeb5-822d3b13822a None admin] PATH_INFO: `/v3/roles/member` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:51.385813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475dbd8-91ab-49f1-aeb5-822d3b13822a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:51.395948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-c475dbd8-91ab-49f1-aeb5-822d3b13822a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:51.395948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:51.395948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:51.395948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:51.395948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:51.395948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:51.395948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:51.395948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:51.395948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:51.395948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:51.395948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:51.395948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:51.395948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:51.395948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:51.395948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:51.395948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:51.396901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:51.396901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:51.396901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:51.396901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:51.396901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:51.396901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:51.396901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:51.396901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:51.396901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:51.396901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:51.396901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:51.396901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:51.396901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:51.396901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:51.396901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:51.396901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:51.397742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:51.397742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:51.397742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:51.397742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:51.397742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:51.397742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:51.397742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:51.397742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:51.397742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:51.397742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:51.397742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:51.397742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:51.397742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 194/395] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:51.403163 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475dbd8-91ab-49f1-aeb5-822d3b13822a None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:51.460362 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:51.485406 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b49c0f-1407-4b12-bd34-1ff348361a5e None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:51.530802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c816a2-a371-48cf-89dc-726633fee537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:51.531522 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c816a2-a371-48cf-89dc-726633fee537 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:51.531866 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c816a2-a371-48cf-89dc-726633fee537 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:51.532227 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c816a2-a371-48cf-89dc-726633fee537 None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:51.533680 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c816a2-a371-48cf-89dc-726633fee537 None admin] RBAC: Authorizing `identity:get_project(project_id=3790dd61247c450ebbf3ef2e90a818e0)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:51.534860 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475dbd8-91ab-49f1-aeb5-822d3b13822a None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:51.535478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c816a2-a371-48cf-89dc-726633fee537 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:51.537831 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 202/396] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:51 2022] GET /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:51.539073 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:51.544889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c816a2-a371-48cf-89dc-726633fee537 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:51.575659 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3184a2-f9f8-4ad0-af98-21c1a26093ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:51.576796 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3184a2-f9f8-4ad0-af98-21c1a26093ef None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:51.577392 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3184a2-f9f8-4ad0-af98-21c1a26093ef None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:51.577839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3184a2-f9f8-4ad0-af98-21c1a26093ef None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:51.578839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3184a2-f9f8-4ad0-af98-21c1a26093ef None admin] RBAC: Adding query filter params (name=member) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:51.579316 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3184a2-f9f8-4ad0-af98-21c1a26093ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:51.580933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3184a2-f9f8-4ad0-af98-21c1a26093ef None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:51.592166 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 195/397] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:51.598071 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3184a2-f9f8-4ad0-af98-21c1a26093ef None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:51.710085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3184a2-f9f8-4ad0-af98-21c1a26093ef None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:51.748290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d4349-0bb0-453f-a9e6-7ef55b3b2fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:51.749663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d4349-0bb0-453f-a9e6-7ef55b3b2fe6 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:51.750160 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d4349-0bb0-453f-a9e6-7ef55b3b2fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:51.750640 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d4349-0bb0-453f-a9e6-7ef55b3b2fe6 None admin] PATH_INFO: `/v3/groups/8a1fb47776af46f6a4e08488588647d9` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:51.752238 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d4349-0bb0-453f-a9e6-7ef55b3b2fe6 None admin] RBAC: Authorizing `identity:get_group(group_id=8a1fb47776af46f6a4e08488588647d9)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:51.753987 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d4349-0bb0-453f-a9e6-7ef55b3b2fe6 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:51.754639 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c816a2-a371-48cf-89dc-726633fee537 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:51.756986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 196/398] 10.222.0.71 () {56 vars in 1180 bytes} [Thu Jul 21 20:25:51 2022] GET /identity/v3/groups/8a1fb47776af46f6a4e08488588647d9 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:51.762575 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d4349-0bb0-453f-a9e6-7ef55b3b2fe6 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:51.781449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:51.826912 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526b3ce-fac1-460f-9867-712296d1753b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:51.828215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526b3ce-fac1-460f-9867-712296d1753b None admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:51.828733 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526b3ce-fac1-460f-9867-712296d1753b None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:51.829236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526b3ce-fac1-460f-9867-712296d1753b None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0/groups/8a1fb47776af46f6a4e08488588647d9/roles/bad7164dd96e4048bab9360ef3f6769e` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:51.835605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526b3ce-fac1-460f-9867-712296d1753b None admin] RBAC: Authorizing `identity:create_grant(project_id=3790dd61247c450ebbf3ef2e90a818e0, group_id=8a1fb47776af46f6a4e08488588647d9, role_id=bad7164dd96e4048bab9360ef3f6769e)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:51.837357 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526b3ce-fac1-460f-9867-712296d1753b None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:51.864472 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 203/399] 10.222.0.71 () {58 vars in 1442 bytes} [Thu Jul 21 20:25:51 2022] PUT /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0/groups/8a1fb47776af46f6a4e08488588647d9/roles/bad7164dd96e4048bab9360ef3f6769e => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:25:51.924325 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d4349-0bb0-453f-a9e6-7ef55b3b2fe6 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:51.964998 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9dd7b-f5f7-41f9-ae58-d66cb140151c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:51.967522 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9dd7b-f5f7-41f9-ae58-d66cb140151c None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:51.968043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9dd7b-f5f7-41f9-ae58-d66cb140151c None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:51.968604 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9dd7b-f5f7-41f9-ae58-d66cb140151c None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:51.970442 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9dd7b-f5f7-41f9-ae58-d66cb140151c None admin] RBAC: Authorizing `identity:get_project(project_id=956f270e083e4b15b70833346dcb8e19)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:51.972206 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9dd7b-f5f7-41f9-ae58-d66cb140151c None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:51.974924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 197/400] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:51 2022] GET /identity/v3/projects/956f270e083e4b15b70833346dcb8e19 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:51.981117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526b3ce-fac1-460f-9867-712296d1753b None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:52.009114 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:52.096464 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526b3ce-fac1-460f-9867-712296d1753b None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:52.137625 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fa704-095c-454d-a6b2-772356516b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:52.139519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fa704-095c-454d-a6b2-772356516b04 None admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:52.140008 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fa704-095c-454d-a6b2-772356516b04 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:52.140489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fa704-095c-454d-a6b2-772356516b04 None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19/groups/8a1fb47776af46f6a4e08488588647d9/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:52.142750 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fa704-095c-454d-a6b2-772356516b04 None admin] RBAC: Authorizing `identity:create_grant(project_id=956f270e083e4b15b70833346dcb8e19, group_id=8a1fb47776af46f6a4e08488588647d9, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:52.144441 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fa704-095c-454d-a6b2-772356516b04 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:52.161730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 204/401] 10.222.0.71 () {58 vars in 1442 bytes} [Thu Jul 21 20:25:51 2022] PUT /identity/v3/projects/956f270e083e4b15b70833346dcb8e19/groups/8a1fb47776af46f6a4e08488588647d9/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:25:52.389822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e22137-07cb-4f27-a886-b4141b9547ec None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:52.390185 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e22137-07cb-4f27-a886-b4141b9547ec None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:52.390361 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e22137-07cb-4f27-a886-b4141b9547ec None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:52.391944 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 198/402] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:52.397903 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ce2d0-ea9a-4789-b14b-0b26aec833a9 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:52.398182 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ce2d0-ea9a-4789-b14b-0b26aec833a9 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:52.398465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ce2d0-ea9a-4789-b14b-0b26aec833a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:52.418686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-b73ce2d0-ea9a-4789-b14b-0b26aec833a9 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:52.420461 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-b73ce2d0-ea9a-4789-b14b-0b26aec833a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:52.542666 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 205/403] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:52.578032 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e22137-07cb-4f27-a886-b4141b9547ec None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:52.721441 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:52.793202 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e22137-07cb-4f27-a886-b4141b9547ec None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:52.874466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e0178-43d2-4b79-b0db-2cf7e5327cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:52.877173 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e0178-43d2-4b79-b0db-2cf7e5327cfc None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:52.877173 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e0178-43d2-4b79-b0db-2cf7e5327cfc None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:52.877173 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e0178-43d2-4b79-b0db-2cf7e5327cfc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:52.888892 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e0178-43d2-4b79-b0db-2cf7e5327cfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:52.897434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-6a1e0178-43d2-4b79-b0db-2cf7e5327cfc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:52.897434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:52.897434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:52.897434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:52.897434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:52.897434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:52.897434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:52.897434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:52.897434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:52.897434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:52.897434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:52.897434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:52.897434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:52.897434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:52.897434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:52.897434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:52.898581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:52.898581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:52.898581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:52.898581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:52.898581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:52.898581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:52.898581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:52.898581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:52.898581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:52.898581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:52.898581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:52.898581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:52.898581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:52.898581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:52.898581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:52.898581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:52.899389 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:52.899389 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:52.899389 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:52.899389 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:52.899389 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:52.899389 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:52.899389 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:52.899389 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:52.899389 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:52.899389 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:52.899389 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:52.899389 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:52.902063 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 199/404] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:25:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:52.910837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ce2d0-ea9a-4789-b14b-0b26aec833a9 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:52.937460 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:53.004322 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6c0f5-a737-48c9-ae19-ff627c4ef9a3 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:53.004806 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6c0f5-a737-48c9-ae19-ff627c4ef9a3 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:53.004806 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6c0f5-a737-48c9-ae19-ff627c4ef9a3 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:53.006251 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 200/405] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:25:53.015319 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b1a55-b2e1-449c-b65c-b295f149d30f None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:53.015608 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b1a55-b2e1-449c-b65c-b295f149d30f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:53.015892 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b1a55-b2e1-449c-b65c-b295f149d30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:53.040596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-1a7b1a55-b2e1-449c-b65c-b295f149d30f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:53.041835 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-1a7b1a55-b2e1-449c-b65c-b295f149d30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:53.127349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ce2d0-ea9a-4789-b14b-0b26aec833a9 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:53.145120 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 201/406] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:53.163986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b1a55-b2e1-449c-b65c-b295f149d30f None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:53.204096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6167a6-4639-42e7-adae-0d1c150c498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:53.204783 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6167a6-4639-42e7-adae-0d1c150c498f None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:53.205062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6167a6-4639-42e7-adae-0d1c150c498f None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:53.205368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6167a6-4639-42e7-adae-0d1c150c498f None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:53.206133 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6167a6-4639-42e7-adae-0d1c150c498f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:53.206442 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6167a6-4639-42e7-adae-0d1c150c498f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:53.207841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6167a6-4639-42e7-adae-0d1c150c498f None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:53.229568 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 206/407] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:25:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:53.230237 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:53.232335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6167a6-4639-42e7-adae-0d1c150c498f None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:53.274777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b1a55-b2e1-449c-b65c-b295f149d30f None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:53.367529 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435ed81-7684-48f0-917d-ef9ab5132197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:53.368358 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435ed81-7684-48f0-917d-ef9ab5132197 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:53.372689 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435ed81-7684-48f0-917d-ef9ab5132197 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:53.372689 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435ed81-7684-48f0-917d-ef9ab5132197 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:53.414558 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435ed81-7684-48f0-917d-ef9ab5132197 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:53.418520 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:53.439537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-9435ed81-7684-48f0-917d-ef9ab5132197 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:53.439537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:53.439537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:53.439537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:53.439537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:53.439537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:53.439537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:53.439537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:53.439537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:53.439537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:53.439537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:53.439537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:53.439537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:53.439537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:53.439537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:53.439537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:53.440510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:53.440510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:53.440510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:53.440510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:53.440510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:53.440510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:53.440510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:53.440510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:53.440510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:53.440510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:53.440510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:53.440510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:53.440510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:53.440510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:53.440510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:53.440510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:53.441383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:53.441383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:53.441383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:53.441383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:53.441383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:53.441383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:53.441383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:53.441383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:53.441383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:53.441383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:53.441383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:53.441383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:53.445141 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 202/408] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:25:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:53.458555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435ed81-7684-48f0-917d-ef9ab5132197 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:53.498892 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6167a6-4639-42e7-adae-0d1c150c498f None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:53.541734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2620d-8a58-4844-b71a-95cf0519845c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:53.542958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2620d-8a58-4844-b71a-95cf0519845c None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:53.542958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2620d-8a58-4844-b71a-95cf0519845c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:53.543130 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2620d-8a58-4844-b71a-95cf0519845c None admin] PATH_INFO: `/v3/groups/8a1fb47776af46f6a4e08488588647d9` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:53.544952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2620d-8a58-4844-b71a-95cf0519845c None admin] RBAC: Authorizing `identity:get_group(group_id=8a1fb47776af46f6a4e08488588647d9)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:53.546535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2620d-8a58-4844-b71a-95cf0519845c None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:53.548459 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 207/409] 10.222.0.71 () {56 vars in 1180 bytes} [Thu Jul 21 20:25:53 2022] GET /identity/v3/groups/8a1fb47776af46f6a4e08488588647d9 => generated 226 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:53.549900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbd68c-8079-4dde-b392-dafeaf2c229d None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:53.550183 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbd68c-8079-4dde-b392-dafeaf2c229d None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:53.550462 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbd68c-8079-4dde-b392-dafeaf2c229d None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:53.552272 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 208/410] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:53.553638 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbd68c-8079-4dde-b392-dafeaf2c229d None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:53.576897 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435ed81-7684-48f0-917d-ef9ab5132197 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:53.600378 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:53.613512 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bf610-c7a2-4f0b-85e7-0d30547f081e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:53.614523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bf610-c7a2-4f0b-85e7-0d30547f081e None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:53.614957 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bf610-c7a2-4f0b-85e7-0d30547f081e None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:53.615424 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bf610-c7a2-4f0b-85e7-0d30547f081e None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:53.616356 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99bf610-c7a2-4f0b-85e7-0d30547f081e None admin] RBAC: Adding query filter params (name=admin) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:53.616890 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99bf610-c7a2-4f0b-85e7-0d30547f081e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:53.618611 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99bf610-c7a2-4f0b-85e7-0d30547f081e None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:53.632863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 203/411] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:53.634496 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4f500-a594-4b30-9c87-ba9ae49bfa51 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:53.634960 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4f500-a594-4b30-9c87-ba9ae49bfa51 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:53.635399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4f500-a594-4b30-9c87-ba9ae49bfa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:53.652388 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-dec4f500-a594-4b30-9c87-ba9ae49bfa51 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:53.654478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-dec4f500-a594-4b30-9c87-ba9ae49bfa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:53.667956 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbd68c-8079-4dde-b392-dafeaf2c229d None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:53.704747 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc181c9-1371-411a-9222-a1133b3271db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:53.706161 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc181c9-1371-411a-9222-a1133b3271db None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:53.706308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc181c9-1371-411a-9222-a1133b3271db None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:53.706524 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc181c9-1371-411a-9222-a1133b3271db None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:53.707845 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc181c9-1371-411a-9222-a1133b3271db None admin] RBAC: Authorizing `identity:get_project(project_id=956f270e083e4b15b70833346dcb8e19)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:53.709578 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc181c9-1371-411a-9222-a1133b3271db None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:53.711936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 209/412] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:53 2022] GET /identity/v3/projects/956f270e083e4b15b70833346dcb8e19 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:53.712930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc181c9-1371-411a-9222-a1133b3271db None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:53.727359 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:53.730929 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 204/413] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:53.733406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4f500-a594-4b30-9c87-ba9ae49bfa51 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:53.827802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc181c9-1371-411a-9222-a1133b3271db None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:53.844551 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4f500-a594-4b30-9c87-ba9ae49bfa51 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:53.860879 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:53.868131 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e2e61-0283-49da-95bf-928463daae42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:53.869105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e2e61-0283-49da-95bf-928463daae42 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:53.869958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e2e61-0283-49da-95bf-928463daae42 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:53.870471 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e2e61-0283-49da-95bf-928463daae42 None admin] PATH_INFO: `/v3/groups/1151069a2c68436ebefc14ce92ef935f` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:53.872818 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e2e61-0283-49da-95bf-928463daae42 None admin] RBAC: Authorizing `identity:get_group(group_id=1151069a2c68436ebefc14ce92ef935f)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:53.874496 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e2e61-0283-49da-95bf-928463daae42 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:53.881120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 210/414] 10.222.0.71 () {56 vars in 1180 bytes} [Thu Jul 21 20:25:53 2022] GET /identity/v3/groups/1151069a2c68436ebefc14ce92ef935f => generated 229 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:53.881120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e2e61-0283-49da-95bf-928463daae42 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:53.892188 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb00e9d-1c98-4bc8-a8cd-d5bb38b2967b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:53.893488 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb00e9d-1c98-4bc8-a8cd-d5bb38b2967b None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:53.894072 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb00e9d-1c98-4bc8-a8cd-d5bb38b2967b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:53.894554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb00e9d-1c98-4bc8-a8cd-d5bb38b2967b None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19/groups/8a1fb47776af46f6a4e08488588647d9/roles/bad7164dd96e4048bab9360ef3f6769e` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:53.897227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb00e9d-1c98-4bc8-a8cd-d5bb38b2967b None admin] RBAC: Authorizing `identity:create_grant(project_id=956f270e083e4b15b70833346dcb8e19, group_id=8a1fb47776af46f6a4e08488588647d9, role_id=bad7164dd96e4048bab9360ef3f6769e)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:53.899081 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb00e9d-1c98-4bc8-a8cd-d5bb38b2967b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:53.914900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 205/415] 10.222.0.71 () {58 vars in 1442 bytes} [Thu Jul 21 20:25:53 2022] PUT /identity/v3/projects/956f270e083e4b15b70833346dcb8e19/groups/8a1fb47776af46f6a4e08488588647d9/roles/bad7164dd96e4048bab9360ef3f6769e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:25:53.916274 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb00e9d-1c98-4bc8-a8cd-d5bb38b2967b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:53.971669 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:53.996489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e2e61-0283-49da-95bf-928463daae42 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:54.035533 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1305cb-84fa-41ed-a1ac-9fc44bddd401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:54.036444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1305cb-84fa-41ed-a1ac-9fc44bddd401 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:54.036735 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1305cb-84fa-41ed-a1ac-9fc44bddd401 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:54.037027 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1305cb-84fa-41ed-a1ac-9fc44bddd401 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:54.047623 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb00e9d-1c98-4bc8-a8cd-d5bb38b2967b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:54.048781 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1305cb-84fa-41ed-a1ac-9fc44bddd401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:54.056886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-0f1305cb-84fa-41ed-a1ac-9fc44bddd401 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:54.056886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:54.056886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:54.056886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:54.056886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:54.056886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:54.056886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:54.056886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:54.056886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:54.056886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:54.056886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:54.056886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:54.056886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:54.056886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:54.056886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:54.056886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:54.058058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:54.058058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:54.058058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:54.058058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:54.058058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:54.058058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:54.058058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:54.058058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:54.058058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:54.058058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:54.058058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:54.058058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:54.058058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:54.058058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:54.058058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:54.058058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:54.058900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:54.058900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:54.058900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:54.058900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:54.058900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:54.058900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:54.058900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:54.058900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:54.058900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:54.058900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:54.058900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:54.058900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:54.058900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 211/416] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:54.070340 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1305cb-84fa-41ed-a1ac-9fc44bddd401 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:54.102882 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebee2a-318d-4dd7-b278-2dc240484517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:54.104346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebee2a-318d-4dd7-b278-2dc240484517 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:54.104853 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebee2a-318d-4dd7-b278-2dc240484517 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:54.105424 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebee2a-318d-4dd7-b278-2dc240484517 None admin] PATH_INFO: `/v3/projects/3dd46499ce284e948f969ddbeb31196c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:54.107696 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ebee2a-318d-4dd7-b278-2dc240484517 None admin] RBAC: Authorizing `identity:get_project(project_id=3dd46499ce284e948f969ddbeb31196c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:54.109555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ebee2a-318d-4dd7-b278-2dc240484517 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:54.111835 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 206/417] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:53 2022] GET /identity/v3/projects/3dd46499ce284e948f969ddbeb31196c => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:54.119093 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebee2a-318d-4dd7-b278-2dc240484517 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:54.197280 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1305cb-84fa-41ed-a1ac-9fc44bddd401 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:54.204520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:54.240716 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2517cb-4288-4f7a-9a98-3dc55c68e0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:54.241431 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2517cb-4288-4f7a-9a98-3dc55c68e0ac None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:54.241715 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2517cb-4288-4f7a-9a98-3dc55c68e0ac None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:54.241992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2517cb-4288-4f7a-9a98-3dc55c68e0ac None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:54.242856 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2517cb-4288-4f7a-9a98-3dc55c68e0ac None admin] RBAC: Adding query filter params (name=member) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:54.243165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2517cb-4288-4f7a-9a98-3dc55c68e0ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:54.245457 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2517cb-4288-4f7a-9a98-3dc55c68e0ac None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:54.251686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:54.256721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 212/418] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:54.263626 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2517cb-4288-4f7a-9a98-3dc55c68e0ac None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:54.269412 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebee2a-318d-4dd7-b278-2dc240484517 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:54.316725 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405d079-749e-4c9f-ad5a-1e571738d438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:54.317662 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405d079-749e-4c9f-ad5a-1e571738d438 None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:54.317947 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405d079-749e-4c9f-ad5a-1e571738d438 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:54.318230 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405d079-749e-4c9f-ad5a-1e571738d438 None admin] PATH_INFO: `/v3/projects/3dd46499ce284e948f969ddbeb31196c/groups/1151069a2c68436ebefc14ce92ef935f/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:54.320572 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405d079-749e-4c9f-ad5a-1e571738d438 None admin] RBAC: Authorizing `identity:create_grant(project_id=3dd46499ce284e948f969ddbeb31196c, group_id=1151069a2c68436ebefc14ce92ef935f, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:54.322099 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405d079-749e-4c9f-ad5a-1e571738d438 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:54.337726 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 207/419] 10.222.0.71 () {58 vars in 1442 bytes} [Thu Jul 21 20:25:54 2022] PUT /identity/v3/projects/3dd46499ce284e948f969ddbeb31196c/groups/1151069a2c68436ebefc14ce92ef935f/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:25:54.382092 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2517cb-4288-4f7a-9a98-3dc55c68e0ac None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:54.438105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b5f6b-2132-469e-940c-f54dc3a1037f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:54.438760 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b5f6b-2132-469e-940c-f54dc3a1037f None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:54.439042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b5f6b-2132-469e-940c-f54dc3a1037f None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:54.439319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b5f6b-2132-469e-940c-f54dc3a1037f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:54.468685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90f1f3-847d-4646-979e-d60aa8711d03 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:54.469006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90f1f3-847d-4646-979e-d60aa8711d03 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:54.469288 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90f1f3-847d-4646-979e-d60aa8711d03 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:54.471820 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 208/420] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:25:54.478611 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612a56e-0abd-4945-8b67-8883eb4eecf8 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:54.478899 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612a56e-0abd-4945-8b67-8883eb4eecf8 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:54.479180 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612a56e-0abd-4945-8b67-8883eb4eecf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:54.522316 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-3612a56e-0abd-4945-8b67-8883eb4eecf8 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:54.523580 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-3612a56e-0abd-4945-8b67-8883eb4eecf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:54.531890 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:54.538331 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b5f6b-2132-469e-940c-f54dc3a1037f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:54.540329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b5f6b-2132-469e-940c-f54dc3a1037f None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:54.614451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 209/421] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:54.642045 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612a56e-0abd-4945-8b67-8883eb4eecf8 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:54.674133 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 213/422] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:54.691364 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b5f6b-2132-469e-940c-f54dc3a1037f None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:54.698567 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:54.827387 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612a56e-0abd-4945-8b67-8883eb4eecf8 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:54.829362 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:54.833059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b5f6b-2132-469e-940c-f54dc3a1037f None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:54.869617 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ca07d-2f64-4b7a-b737-9b23a6efe3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:54.870487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ca07d-2f64-4b7a-b737-9b23a6efe3d5 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:54.870767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ca07d-2f64-4b7a-b737-9b23a6efe3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:54.871075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ca07d-2f64-4b7a-b737-9b23a6efe3d5 None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:54.872344 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ca07d-2f64-4b7a-b737-9b23a6efe3d5 None admin] RBAC: Authorizing `identity:get_project(project_id=3790dd61247c450ebbf3ef2e90a818e0)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:54.873888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ca07d-2f64-4b7a-b737-9b23a6efe3d5 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:54.875613 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 214/423] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:54 2022] GET /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:54.881944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ca07d-2f64-4b7a-b737-9b23a6efe3d5 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:54.925811 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926c926-03f5-40b9-a6ca-b655b0161db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:54.926652 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926c926-03f5-40b9-a6ca-b655b0161db6 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:54.926937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926c926-03f5-40b9-a6ca-b655b0161db6 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:54.927215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926c926-03f5-40b9-a6ca-b655b0161db6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:54.947290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926c926-03f5-40b9-a6ca-b655b0161db6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:54.964676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-8926c926-03f5-40b9-a6ca-b655b0161db6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:54.964676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:54.964676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:54.964676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:54.964676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:54.964676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:54.964676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:54.964676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:54.964676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:54.964676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:54.964676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:54.964676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:54.964676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:54.964676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:54.964676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:54.964676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:54.965686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:54.965686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:54.965686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:54.965686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:54.965686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:54.965686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:54.965686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:54.965686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:54.965686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:54.965686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:54.965686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:54.965686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:54.965686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:54.965686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:54.965686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:54.965686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:54.967080 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:54.967080 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:54.967080 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:54.967080 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:54.967080 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:54.967080 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:54.967080 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:54.967080 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:54.967080 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:54.967080 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:54.967080 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:54.967080 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:54.967080 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 210/424] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:25:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:25:54.975570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926c926-03f5-40b9-a6ca-b655b0161db6 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:55.006514 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ca07d-2f64-4b7a-b737-9b23a6efe3d5 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:55.047956 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048f681-22b8-4834-b0bc-a0d78614368d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:55.049289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048f681-22b8-4834-b0bc-a0d78614368d None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:55.049800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048f681-22b8-4834-b0bc-a0d78614368d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:55.050278 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048f681-22b8-4834-b0bc-a0d78614368d None admin] PATH_INFO: `/v3/groups/8a1fb47776af46f6a4e08488588647d9` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:55.051744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048f681-22b8-4834-b0bc-a0d78614368d None admin] RBAC: Authorizing `identity:get_group(group_id=8a1fb47776af46f6a4e08488588647d9)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:55.053451 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048f681-22b8-4834-b0bc-a0d78614368d None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:55.055747 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 215/425] 10.222.0.71 () {56 vars in 1180 bytes} [Thu Jul 21 20:25:54 2022] GET /identity/v3/groups/8a1fb47776af46f6a4e08488588647d9 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:55.062126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976df2e-1f2d-4bf9-a816-585adda0aab8 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:55.062795 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976df2e-1f2d-4bf9-a816-585adda0aab8 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:55.063301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976df2e-1f2d-4bf9-a816-585adda0aab8 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:55.065131 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 216/426] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:55.066105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976df2e-1f2d-4bf9-a816-585adda0aab8 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:55.106232 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:55.116638 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:55.129501 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926c926-03f5-40b9-a6ca-b655b0161db6 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:55.177533 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcd8d7-7bae-4643-9b54-680d26c0d3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:55.178898 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcd8d7-7bae-4643-9b54-680d26c0d3ea None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:55.179399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcd8d7-7bae-4643-9b54-680d26c0d3ea None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:55.179900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcd8d7-7bae-4643-9b54-680d26c0d3ea None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:55.180993 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcd8d7-7bae-4643-9b54-680d26c0d3ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:55.181644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcd8d7-7bae-4643-9b54-680d26c0d3ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:55.183409 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcd8d7-7bae-4643-9b54-680d26c0d3ea None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:55.186555 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976df2e-1f2d-4bf9-a816-585adda0aab8 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:55.199308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 211/427] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:25:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:55.201199 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf86ed7-039f-4b39-b134-514f1c2e11e2 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:55.201749 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf86ed7-039f-4b39-b134-514f1c2e11e2 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:55.202326 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf86ed7-039f-4b39-b134-514f1c2e11e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:55.222888 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-6bf86ed7-039f-4b39-b134-514f1c2e11e2 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:55.224805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-6bf86ed7-039f-4b39-b134-514f1c2e11e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:55.228965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f0185-7eed-47fe-af48-4fd5fec774b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:55.229335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f0185-7eed-47fe-af48-4fd5fec774b7 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:55.229508 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f0185-7eed-47fe-af48-4fd5fec774b7 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:55.229788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f0185-7eed-47fe-af48-4fd5fec774b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:55.230650 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f0185-7eed-47fe-af48-4fd5fec774b7 None admin] RBAC: Adding query filter params (group.id=8a1fb47776af46f6a4e08488588647d9, role.id=8a0eeb2a21dc4897a58fb5851d58678c, scope.project.id=3790dd61247c450ebbf3ef2e90a818e0) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:55.230967 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f0185-7eed-47fe-af48-4fd5fec774b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:55.232587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f0185-7eed-47fe-af48-4fd5fec774b7 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:55.245225 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 217/428] 10.222.0.71 () {56 vars in 1376 bytes} [Thu Jul 21 20:25:55 2022] GET /identity/v3/role_assignments?group.id=8a1fb47776af46f6a4e08488588647d9&scope.project.id=3790dd61247c450ebbf3ef2e90a818e0&role.id=8a0eeb2a21dc4897a58fb5851d58678c => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:55.246261 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f0185-7eed-47fe-af48-4fd5fec774b7 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:55.309663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 212/429] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:55.338218 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf86ed7-039f-4b39-b134-514f1c2e11e2 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:55.484987 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:55.492184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f0185-7eed-47fe-af48-4fd5fec774b7 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:55.516700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:55.538858 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf86ed7-039f-4b39-b134-514f1c2e11e2 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:55.545037 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0719e0a-4d64-4500-8df7-e0501816030b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:55.546168 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0719e0a-4d64-4500-8df7-e0501816030b None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:55.546702 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0719e0a-4d64-4500-8df7-e0501816030b None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:55.547202 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0719e0a-4d64-4500-8df7-e0501816030b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:55.594098 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea538dee-286d-4f72-a4cb-f4b6e2f69b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:55.595722 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea538dee-286d-4f72-a4cb-f4b6e2f69b0a None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:55.596009 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea538dee-286d-4f72-a4cb-f4b6e2f69b0a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:55.596284 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea538dee-286d-4f72-a4cb-f4b6e2f69b0a None admin] PATH_INFO: `/v3/roles/member` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:55.614090 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea538dee-286d-4f72-a4cb-f4b6e2f69b0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:55.629269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-ea538dee-286d-4f72-a4cb-f4b6e2f69b0a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:55.629269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:55.629269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:55.629269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:55.629269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:55.629269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:55.629269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:55.629269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:55.629269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:55.629269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:55.629269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:55.629269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:55.629269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:55.629269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:55.629269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:55.629269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:55.630253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:55.630253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:55.630253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:55.630253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:55.630253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:55.630253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:55.630253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:55.630253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:55.630253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:55.630253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:55.630253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:55.630253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:55.630253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:55.630253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:55.630253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:55.630253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:55.631085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:55.631085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:55.631085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:55.631085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:55.631085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:55.631085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:55.631085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:55.631085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:55.631085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:55.631085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:55.631085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:25:55.631085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:55.639750 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 213/430] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:55.647226 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea538dee-286d-4f72-a4cb-f4b6e2f69b0a None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:55.698193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0719e0a-4d64-4500-8df7-e0501816030b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:55.704247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0719e0a-4d64-4500-8df7-e0501816030b None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:55.776259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea538dee-286d-4f72-a4cb-f4b6e2f69b0a None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:55.838767 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288d73c-a843-47b0-8876-f6b2f88698d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:55.839494 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288d73c-a843-47b0-8876-f6b2f88698d6 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:55.839787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288d73c-a843-47b0-8876-f6b2f88698d6 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:55.840885 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288d73c-a843-47b0-8876-f6b2f88698d6 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:55.841775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e288d73c-a843-47b0-8876-f6b2f88698d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:55.842097 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e288d73c-a843-47b0-8876-f6b2f88698d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:55.844415 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e288d73c-a843-47b0-8876-f6b2f88698d6 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:55.857571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 214/431] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:25:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:55.867139 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288d73c-a843-47b0-8876-f6b2f88698d6 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:55.884376 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:55.893642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:55.900522 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 218/432] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 655 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:55.911003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0719e0a-4d64-4500-8df7-e0501816030b None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:56.018456 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288d73c-a843-47b0-8876-f6b2f88698d6 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:56.039960 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0719e0a-4d64-4500-8df7-e0501816030b None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:56.057028 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec87dec-7273-4e5c-979f-abf52d18dfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:56.057932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec87dec-7273-4e5c-979f-abf52d18dfbc None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:56.058045 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec87dec-7273-4e5c-979f-abf52d18dfbc None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:56.058271 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec87dec-7273-4e5c-979f-abf52d18dfbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:56.081818 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f497b75-3d21-4370-8371-812cc7e4dac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:56.083071 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f497b75-3d21-4370-8371-812cc7e4dac4 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:56.083659 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f497b75-3d21-4370-8371-812cc7e4dac4 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:56.084146 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f497b75-3d21-4370-8371-812cc7e4dac4 None admin] PATH_INFO: `/v3/projects/3790dd61247c450ebbf3ef2e90a818e0` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:56.085819 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f497b75-3d21-4370-8371-812cc7e4dac4 None admin] RBAC: Authorizing `identity:get_project(project_id=3790dd61247c450ebbf3ef2e90a818e0)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:56.087704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f497b75-3d21-4370-8371-812cc7e4dac4 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:56.090101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 219/433] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:55 2022] GET /identity/v3/projects/3790dd61247c450ebbf3ef2e90a818e0 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:56.098329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f497b75-3d21-4370-8371-812cc7e4dac4 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:56.143495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec87dec-7273-4e5c-979f-abf52d18dfbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:56.144733 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec87dec-7273-4e5c-979f-abf52d18dfbc None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:56.154089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:56.199822 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:56.252637 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f497b75-3d21-4370-8371-812cc7e4dac4 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:56.270812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 215/434] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:56.277652 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec87dec-7273-4e5c-979f-abf52d18dfbc None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:56.296586 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a601c8c-d02f-4663-9db1-41dba06fc2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:56.297583 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a601c8c-d02f-4663-9db1-41dba06fc2f5 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:56.297876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a601c8c-d02f-4663-9db1-41dba06fc2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:56.298152 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a601c8c-d02f-4663-9db1-41dba06fc2f5 None admin] PATH_INFO: `/v3/groups/8a1fb47776af46f6a4e08488588647d9` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:56.302881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a601c8c-d02f-4663-9db1-41dba06fc2f5 None admin] RBAC: Authorizing `identity:get_group(group_id=8a1fb47776af46f6a4e08488588647d9)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:56.304591 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a601c8c-d02f-4663-9db1-41dba06fc2f5 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:56.306855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 220/435] 10.222.0.71 () {56 vars in 1180 bytes} [Thu Jul 21 20:25:56 2022] GET /identity/v3/groups/8a1fb47776af46f6a4e08488588647d9 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:56.312669 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a601c8c-d02f-4663-9db1-41dba06fc2f5 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:56.407632 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec87dec-7273-4e5c-979f-abf52d18dfbc None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:56.416752 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:56.426664 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a601c8c-d02f-4663-9db1-41dba06fc2f5 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:56.451458 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b7a3e-a6e5-4350-b243-100a8a4d95f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:56.452808 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b7a3e-a6e5-4350-b243-100a8a4d95f7 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:56.453428 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b7a3e-a6e5-4350-b243-100a8a4d95f7 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:56.453937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b7a3e-a6e5-4350-b243-100a8a4d95f7 None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:56.455450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b7a3e-a6e5-4350-b243-100a8a4d95f7 None admin] RBAC: Authorizing `identity:get_project(project_id=956f270e083e4b15b70833346dcb8e19)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:56.457370 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b7a3e-a6e5-4350-b243-100a8a4d95f7 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:56.464523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 216/436] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:56 2022] GET /identity/v3/projects/956f270e083e4b15b70833346dcb8e19 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:56.465372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b7a3e-a6e5-4350-b243-100a8a4d95f7 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:56.470308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c915cee-eae7-4ea8-83ae-39fc300e1f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:56.471451 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c915cee-eae7-4ea8-83ae-39fc300e1f77 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:56.471945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c915cee-eae7-4ea8-83ae-39fc300e1f77 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:56.472425 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c915cee-eae7-4ea8-83ae-39fc300e1f77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:56.473528 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c915cee-eae7-4ea8-83ae-39fc300e1f77 None admin] RBAC: Adding query filter params (group.id=8a1fb47776af46f6a4e08488588647d9, role.id=bad7164dd96e4048bab9360ef3f6769e, scope.project.id=3790dd61247c450ebbf3ef2e90a818e0) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:56.474076 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c915cee-eae7-4ea8-83ae-39fc300e1f77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:56.476734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c915cee-eae7-4ea8-83ae-39fc300e1f77 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:56.482268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:56.489662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 221/437] 10.222.0.71 () {56 vars in 1376 bytes} [Thu Jul 21 20:25:56 2022] GET /identity/v3/role_assignments?group.id=8a1fb47776af46f6a4e08488588647d9&scope.project.id=3790dd61247c450ebbf3ef2e90a818e0&role.id=bad7164dd96e4048bab9360ef3f6769e => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:56.602001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b7a3e-a6e5-4350-b243-100a8a4d95f7 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:56.639498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc798c4-0070-4191-996c-3ae8211658f3 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:56.639863 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc798c4-0070-4191-996c-3ae8211658f3 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:56.640059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc798c4-0070-4191-996c-3ae8211658f3 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:56.642012 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 222/438] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:56.648159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bbbdf-af82-4c87-b87e-4ba535db3edd None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:56.648494 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bbbdf-af82-4c87-b87e-4ba535db3edd None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:56.648786 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bbbdf-af82-4c87-b87e-4ba535db3edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:56.650962 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f5f75-38df-4e50-836f-8ba772c796e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:56.652195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f5f75-38df-4e50-836f-8ba772c796e3 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:56.652741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f5f75-38df-4e50-836f-8ba772c796e3 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:56.653284 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f5f75-38df-4e50-836f-8ba772c796e3 None admin] PATH_INFO: `/v3/groups/8a1fb47776af46f6a4e08488588647d9` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:56.655621 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f5f75-38df-4e50-836f-8ba772c796e3 None admin] RBAC: Authorizing `identity:get_group(group_id=8a1fb47776af46f6a4e08488588647d9)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:56.657447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f5f75-38df-4e50-836f-8ba772c796e3 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:56.659810 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 217/439] 10.222.0.71 () {56 vars in 1180 bytes} [Thu Jul 21 20:25:56 2022] GET /identity/v3/groups/8a1fb47776af46f6a4e08488588647d9 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:56.666994 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f5f75-38df-4e50-836f-8ba772c796e3 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:56.669116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-a59bbbdf-af82-4c87-b87e-4ba535db3edd None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:56.670461 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a59bbbdf-af82-4c87-b87e-4ba535db3edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:56.728056 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:56.769992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 223/440] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:25:56.771975 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb5075-e4e4-44f1-b586-0b10f10dadf6 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:56.772269 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb5075-e4e4-44f1-b586-0b10f10dadf6 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:56.772545 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb5075-e4e4-44f1-b586-0b10f10dadf6 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:56.774978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 224/441] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:25:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:25:56.784218 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf362858-e459-4341-b4d8-675632b5a9a1 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:56.784544 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf362858-e459-4341-b4d8-675632b5a9a1 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:56.784783 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf362858-e459-4341-b4d8-675632b5a9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:56.804568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f5f75-38df-4e50-836f-8ba772c796e3 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:56.805993 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-cf362858-e459-4341-b4d8-675632b5a9a1 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:25:56.808120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-cf362858-e459-4341-b4d8-675632b5a9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:25:56.847337 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901d35d-85bb-4532-980e-5b5d6d3ca6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:56.848024 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901d35d-85bb-4532-980e-5b5d6d3ca6f3 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:56.848304 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901d35d-85bb-4532-980e-5b5d6d3ca6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:56.848579 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901d35d-85bb-4532-980e-5b5d6d3ca6f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:56.849740 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901d35d-85bb-4532-980e-5b5d6d3ca6f3 None admin] RBAC: Adding query filter params (group.id=8a1fb47776af46f6a4e08488588647d9, role.id=8a0eeb2a21dc4897a58fb5851d58678c, scope.project.id=956f270e083e4b15b70833346dcb8e19) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:56.850055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901d35d-85bb-4532-980e-5b5d6d3ca6f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:56.851633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901d35d-85bb-4532-980e-5b5d6d3ca6f3 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:56.859156 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 218/442] 10.222.0.71 () {56 vars in 1376 bytes} [Thu Jul 21 20:25:56 2022] GET /identity/v3/role_assignments?group.id=8a1fb47776af46f6a4e08488588647d9&scope.project.id=956f270e083e4b15b70833346dcb8e19&role.id=8a0eeb2a21dc4897a58fb5851d58678c => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:56.859718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901d35d-85bb-4532-980e-5b5d6d3ca6f3 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:56.909099 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 225/443] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:25:56.929061 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf362858-e459-4341-b4d8-675632b5a9a1 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:57.027768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901d35d-85bb-4532-980e-5b5d6d3ca6f3 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:57.038174 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:57.046478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:57.087233 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cbba7-8729-4c40-97ff-b95aac69a986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:57.088081 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cbba7-8729-4c40-97ff-b95aac69a986 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:57.088367 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cbba7-8729-4c40-97ff-b95aac69a986 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:57.088643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cbba7-8729-4c40-97ff-b95aac69a986 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:57.107228 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cbba7-8729-4c40-97ff-b95aac69a986 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:57.112729 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf362858-e459-4341-b4d8-675632b5a9a1 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:57.123509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-cc6cbba7-8729-4c40-97ff-b95aac69a986 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:57.123509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:57.123509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:57.123509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:57.123509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:57.123509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:57.123509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:57.123509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:57.123509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:57.123509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:57.123509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:57.123509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:57.123509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:57.123509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:57.123509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:57.123509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:57.124642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:57.124642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:57.124642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:57.124642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:57.124642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:57.124642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:57.124642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:57.124642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:57.124642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:57.124642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:57.124642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:57.124642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:57.124642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:57.124642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:57.124642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:57.124642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:57.125783 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:57.125783 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:57.125783 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:57.125783 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:57.125783 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:57.125783 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:57.125783 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:57.125783 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:57.125783 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:57.125783 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:57.125783 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:25:57.125783 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:25:57.126539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 219/444] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:25:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:57.135639 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cbba7-8729-4c40-97ff-b95aac69a986 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:57.161043 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a956a03-3ace-4db2-93cb-e3c1af0b818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:57.161887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a956a03-3ace-4db2-93cb-e3c1af0b818d None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:57.162165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a956a03-3ace-4db2-93cb-e3c1af0b818d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:57.162432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a956a03-3ace-4db2-93cb-e3c1af0b818d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:57.177735 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a956a03-3ace-4db2-93cb-e3c1af0b818d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:25:57.186083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-1a956a03-3ace-4db2-93cb-e3c1af0b818d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:57.186083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:25:57.186083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:25:57.186083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:25:57.186083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:25:57.186083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:25:57.186083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:25:57.186083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:25:57.186083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:25:57.186083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:25:57.186083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:25:57.186083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:25:57.186083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:25:57.186083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:25:57.186083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:25:57.186083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:25:57.187042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:25:57.187042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:25:57.187042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:25:57.187042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:25:57.187042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:25:57.187042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:25:57.187042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:25:57.187042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:25:57.187042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:25:57.187042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:25:57.187042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:25:57.187042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:25:57.187042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:25:57.187042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:25:57.187042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:25:57.187042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:25:57.190072 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:25:57.190072 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:25:57.190072 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:25:57.190072 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:25:57.190072 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:25:57.190072 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:25:57.190072 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:25:57.190072 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:25:57.190072 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:25:57.190072 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:25:57.190072 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:25:57.190072 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:25:57.190072 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 226/445] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:25:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:25:57.193809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a956a03-3ace-4db2-93cb-e3c1af0b818d None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:57.311997 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cbba7-8729-4c40-97ff-b95aac69a986 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:57.354971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a956a03-3ace-4db2-93cb-e3c1af0b818d None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:57.358503 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54851e-d95f-43eb-9224-8214cbacc481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:57.359345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54851e-d95f-43eb-9224-8214cbacc481 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:57.359755 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54851e-d95f-43eb-9224-8214cbacc481 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:57.360170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54851e-d95f-43eb-9224-8214cbacc481 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:57.361104 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54851e-d95f-43eb-9224-8214cbacc481 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:57.361539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54851e-d95f-43eb-9224-8214cbacc481 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:57.363133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54851e-d95f-43eb-9224-8214cbacc481 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:57.373031 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 220/446] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:25:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:57.378469 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54851e-d95f-43eb-9224-8214cbacc481 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:57.379043 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:57.395678 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b76b8-e6a5-4995-a63a-6c9e47b19789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:57.396491 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b76b8-e6a5-4995-a63a-6c9e47b19789 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:57.396920 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b76b8-e6a5-4995-a63a-6c9e47b19789 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:57.397328 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b76b8-e6a5-4995-a63a-6c9e47b19789 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:57.398209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b76b8-e6a5-4995-a63a-6c9e47b19789 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:57.398563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b76b8-e6a5-4995-a63a-6c9e47b19789 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:57.400008 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b76b8-e6a5-4995-a63a-6c9e47b19789 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:57.407614 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:57.409329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 227/447] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:25:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:57.415519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b76b8-e6a5-4995-a63a-6c9e47b19789 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:57.490425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54851e-d95f-43eb-9224-8214cbacc481 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:57.526221 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b76b8-e6a5-4995-a63a-6c9e47b19789 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:57.526538 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e21b2-c3a3-4160-ab72-32230dad5a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:57.527716 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e21b2-c3a3-4160-ab72-32230dad5a38 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:57.527716 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e21b2-c3a3-4160-ab72-32230dad5a38 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:57.527716 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e21b2-c3a3-4160-ab72-32230dad5a38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:57.561395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3aa05-8772-4ffb-b934-150030a3a019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:57.562041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3aa05-8772-4ffb-b934-150030a3a019 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:57.562316 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3aa05-8772-4ffb-b934-150030a3a019 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:57.562598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3aa05-8772-4ffb-b934-150030a3a019 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:57.600697 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e21b2-c3a3-4160-ab72-32230dad5a38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:57.602270 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e21b2-c3a3-4160-ab72-32230dad5a38 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:57.614095 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:57.631213 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:57.635237 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3aa05-8772-4ffb-b934-150030a3a019 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:57.636709 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3aa05-8772-4ffb-b934-150030a3a019 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:57.722037 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 221/448] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 20:25:57.728799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e21b2-c3a3-4160-ab72-32230dad5a38 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:57.748369 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 228/449] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:25:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:25:57.754222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3aa05-8772-4ffb-b934-150030a3a019 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:57.854703 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:57.876407 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e21b2-c3a3-4160-ab72-32230dad5a38 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:57.878720 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3aa05-8772-4ffb-b934-150030a3a019 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:57.919270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d55da8-9dda-486a-9d37-2c7296db310c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:57.920138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d55da8-9dda-486a-9d37-2c7296db310c None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:57.920417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d55da8-9dda-486a-9d37-2c7296db310c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:57.920690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d55da8-9dda-486a-9d37-2c7296db310c None admin] PATH_INFO: `/v3/projects/3dd46499ce284e948f969ddbeb31196c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:57.922203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d55da8-9dda-486a-9d37-2c7296db310c None admin] RBAC: Authorizing `identity:get_project(project_id=3dd46499ce284e948f969ddbeb31196c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:57.924044 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d55da8-9dda-486a-9d37-2c7296db310c None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:57.926254 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 229/450] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:57 2022] GET /identity/v3/projects/3dd46499ce284e948f969ddbeb31196c => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:57.928759 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:57.932652 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d55da8-9dda-486a-9d37-2c7296db310c None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:57.943549 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967603e-8230-45cc-ad29-97d84fe867f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:57.945155 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967603e-8230-45cc-ad29-97d84fe867f6 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:57.945663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967603e-8230-45cc-ad29-97d84fe867f6 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:57.946173 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967603e-8230-45cc-ad29-97d84fe867f6 None admin] PATH_INFO: `/v3/projects/956f270e083e4b15b70833346dcb8e19` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:57.948527 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967603e-8230-45cc-ad29-97d84fe867f6 None admin] RBAC: Authorizing `identity:get_project(project_id=956f270e083e4b15b70833346dcb8e19)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:57.951938 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967603e-8230-45cc-ad29-97d84fe867f6 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:57.960013 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 222/451] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:25:57 2022] GET /identity/v3/projects/956f270e083e4b15b70833346dcb8e19 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:25:57.961743 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967603e-8230-45cc-ad29-97d84fe867f6 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:58.070805 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d55da8-9dda-486a-9d37-2c7296db310c None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:58.099023 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967603e-8230-45cc-ad29-97d84fe867f6 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:58.112501 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba950b-dbfe-46d4-be63-51aa2a575364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:58.113400 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba950b-dbfe-46d4-be63-51aa2a575364 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:58.113679 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba950b-dbfe-46d4-be63-51aa2a575364 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:58.113947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba950b-dbfe-46d4-be63-51aa2a575364 None admin] PATH_INFO: `/v3/groups/1151069a2c68436ebefc14ce92ef935f` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:58.115210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba950b-dbfe-46d4-be63-51aa2a575364 None admin] RBAC: Authorizing `identity:get_group(group_id=1151069a2c68436ebefc14ce92ef935f)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:58.116665 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba950b-dbfe-46d4-be63-51aa2a575364 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:58.118359 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 230/452] 10.222.0.71 () {56 vars in 1180 bytes} [Thu Jul 21 20:25:57 2022] GET /identity/v3/groups/1151069a2c68436ebefc14ce92ef935f => generated 229 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:58.125559 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba950b-dbfe-46d4-be63-51aa2a575364 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:58.139773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e4c8d-bd5b-4f58-942c-1658995e19fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:58.140569 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e4c8d-bd5b-4f58-942c-1658995e19fe None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:58.140853 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e4c8d-bd5b-4f58-942c-1658995e19fe None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:58.141164 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e4c8d-bd5b-4f58-942c-1658995e19fe None admin] PATH_INFO: `/v3/groups/8a1fb47776af46f6a4e08488588647d9` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:58.142422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e4c8d-bd5b-4f58-942c-1658995e19fe None admin] RBAC: Authorizing `identity:get_group(group_id=8a1fb47776af46f6a4e08488588647d9)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:58.143895 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e4c8d-bd5b-4f58-942c-1658995e19fe None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:58.145609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 223/453] 10.222.0.71 () {56 vars in 1180 bytes} [Thu Jul 21 20:25:57 2022] GET /identity/v3/groups/8a1fb47776af46f6a4e08488588647d9 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:58.151318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:58.151536 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e4c8d-bd5b-4f58-942c-1658995e19fe None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:25:58.219863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:25:58.235434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba950b-dbfe-46d4-be63-51aa2a575364 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:58.259309 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e4c8d-bd5b-4f58-942c-1658995e19fe None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:25:58.270310 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6f61e-e6ae-47ca-b62c-89138b6b1ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:58.271194 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6f61e-e6ae-47ca-b62c-89138b6b1ae0 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:58.271687 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6f61e-e6ae-47ca-b62c-89138b6b1ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:58.271913 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6f61e-e6ae-47ca-b62c-89138b6b1ae0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:58.272757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6f61e-e6ae-47ca-b62c-89138b6b1ae0 None admin] RBAC: Adding query filter params (group.id=1151069a2c68436ebefc14ce92ef935f, role.id=19f5e72c0be44a1197c7288de82e79e8, scope.project.id=3dd46499ce284e948f969ddbeb31196c) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:58.273101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6f61e-e6ae-47ca-b62c-89138b6b1ae0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:58.274528 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6f61e-e6ae-47ca-b62c-89138b6b1ae0 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:58.280620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 231/454] 10.222.0.71 () {56 vars in 1376 bytes} [Thu Jul 21 20:25:58 2022] GET /identity/v3/role_assignments?group.id=1151069a2c68436ebefc14ce92ef935f&scope.project.id=3dd46499ce284e948f969ddbeb31196c&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:25:58.294651 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cb5ce-1f43-4e9e-a81a-309750d60e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:25:58.295332 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cb5ce-1f43-4e9e-a81a-309750d60e14 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:25:58.298347 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cb5ce-1f43-4e9e-a81a-309750d60e14 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:25:58.298347 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cb5ce-1f43-4e9e-a81a-309750d60e14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:25:58.298347 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01cb5ce-1f43-4e9e-a81a-309750d60e14 None admin] RBAC: Adding query filter params (group.id=8a1fb47776af46f6a4e08488588647d9, role.id=bad7164dd96e4048bab9360ef3f6769e, scope.project.id=956f270e083e4b15b70833346dcb8e19) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:25:58.298347 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01cb5ce-1f43-4e9e-a81a-309750d60e14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:25:58.298755 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01cb5ce-1f43-4e9e-a81a-309750d60e14 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:25:58.306209 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 224/455] 10.222.0.71 () {56 vars in 1376 bytes} [Thu Jul 21 20:25:58 2022] GET /identity/v3/role_assignments?group.id=8a1fb47776af46f6a4e08488588647d9&scope.project.id=956f270e083e4b15b70833346dcb8e19&role.id=bad7164dd96e4048bab9360ef3f6769e => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:00.952802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cbbd9-34be-4c50-82c7-f31a92515a7f None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:00.953359 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cbbd9-34be-4c50-82c7-f31a92515a7f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:00.953359 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cbbd9-34be-4c50-82c7-f31a92515a7f None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:00.955056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 232/456] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:00.960456 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d661ec-e330-416c-8424-c0b57d53df64 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:00.961012 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d661ec-e330-416c-8424-c0b57d53df64 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:00.961012 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d661ec-e330-416c-8424-c0b57d53df64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:00.991133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-59d661ec-e330-416c-8424-c0b57d53df64 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:00.992332 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-59d661ec-e330-416c-8424-c0b57d53df64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:01.075254 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 225/457] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:26:01.112010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cbbd9-34be-4c50-82c7-f31a92515a7f None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:01.135350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa844fe5-c0d3-42a4-88bc-d602baed8a3d None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:01.135679 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa844fe5-c0d3-42a4-88bc-d602baed8a3d None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:01.135975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa844fe5-c0d3-42a4-88bc-d602baed8a3d None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:01.138507 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 226/458] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:01.150872 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ec14d-2732-44cc-a5d2-a720bd738273 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:01.151219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ec14d-2732-44cc-a5d2-a720bd738273 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:01.151432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ec14d-2732-44cc-a5d2-a720bd738273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:01.186268 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-9e5ec14d-2732-44cc-a5d2-a720bd738273 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:01.187656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-9e5ec14d-2732-44cc-a5d2-a720bd738273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:01.223436 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:01.247530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cbbd9-34be-4c50-82c7-f31a92515a7f None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:01.286568 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24302785-a824-467b-b974-1f3bc694c207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:01.287771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24302785-a824-467b-b974-1f3bc694c207 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:01.288261 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24302785-a824-467b-b974-1f3bc694c207 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:01.288754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24302785-a824-467b-b974-1f3bc694c207 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:01.300472 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24302785-a824-467b-b974-1f3bc694c207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:01.302306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24302785-a824-467b-b974-1f3bc694c207 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:01.323167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-24302785-a824-467b-b974-1f3bc694c207 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:01.323167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:01.323167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:01.323167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:01.323167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:01.323167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:01.323167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:01.323167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:01.323167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:01.323167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:01.323167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:01.323167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:01.323167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:01.323167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:01.323167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:01.324107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:01.324107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:01.324107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:01.324107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:01.324107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:01.324107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:01.324107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:01.324107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:01.324107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:01.324107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:01.324107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:01.324107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:01.324107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:01.324107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:01.324107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:01.324107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:01.324107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:01.324964 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:01.324964 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:01.324964 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:01.324964 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:01.327690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 233/459] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:01.335699 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:01.337643 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24302785-a824-467b-b974-1f3bc694c207 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:01.366906 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 227/460] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:01.413239 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ec14d-2732-44cc-a5d2-a720bd738273 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:01.542400 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24302785-a824-467b-b974-1f3bc694c207 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:01.595015 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8589957-c249-4986-9f8f-2cd602061571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:01.595685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8589957-c249-4986-9f8f-2cd602061571 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:01.595966 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8589957-c249-4986-9f8f-2cd602061571 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:01.596238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8589957-c249-4986-9f8f-2cd602061571 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:01.596965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8589957-c249-4986-9f8f-2cd602061571 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:01.597332 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8589957-c249-4986-9f8f-2cd602061571 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:01.598984 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8589957-c249-4986-9f8f-2cd602061571 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:01.601034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:01.608923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ec14d-2732-44cc-a5d2-a720bd738273 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:01.620897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 234/461] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:01.622459 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efc8d8-da4c-4a6e-99f8-bd9e3593fbee None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:01.622752 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efc8d8-da4c-4a6e-99f8-bd9e3593fbee None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:01.623027 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efc8d8-da4c-4a6e-99f8-bd9e3593fbee None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:01.624261 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 235/462] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:01.629300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efc8d8-da4c-4a6e-99f8-bd9e3593fbee None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:01.657315 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c08a8-1e78-46ab-8246-7c1c24c812ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:01.658118 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c08a8-1e78-46ab-8246-7c1c24c812ee None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:01.658394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c08a8-1e78-46ab-8246-7c1c24c812ee None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:01.658660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c08a8-1e78-46ab-8246-7c1c24c812ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:01.671221 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c08a8-1e78-46ab-8246-7c1c24c812ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:01.672708 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c08a8-1e78-46ab-8246-7c1c24c812ee None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:01.700298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-ed6c08a8-1e78-46ab-8246-7c1c24c812ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:01.700298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:01.700298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:01.700298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:01.700298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:01.700298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:01.700298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:01.700298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:01.700298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:01.700298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:01.700298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:01.700298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:01.700298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:01.700298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:01.700298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:01.701303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:01.701303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:01.701303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:01.701303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:01.701303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:01.701303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:01.701303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:01.701303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:01.701303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:01.701303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:01.701303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:01.701303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:01.701303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:01.701303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:01.701303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:01.701303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:01.701303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:01.702168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:01.702168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:01.702168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:01.702168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:01.702937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 228/463] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:01.704001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fc6b2-ec0d-4427-9042-8f5e2b93b75e None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:01.704345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fc6b2-ec0d-4427-9042-8f5e2b93b75e None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:01.704624 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fc6b2-ec0d-4427-9042-8f5e2b93b75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:01.722744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-cb7fc6b2-ec0d-4427-9042-8f5e2b93b75e None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:01.724128 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-cb7fc6b2-ec0d-4427-9042-8f5e2b93b75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:01.762333 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:01.763832 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efc8d8-da4c-4a6e-99f8-bd9e3593fbee None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:01.811990 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae3a50-9c37-469c-a249-cdfda06329e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:01.812971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae3a50-9c37-469c-a249-cdfda06329e4 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:01.813476 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae3a50-9c37-469c-a249-cdfda06329e4 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:01.813929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae3a50-9c37-469c-a249-cdfda06329e4 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:01.814837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae3a50-9c37-469c-a249-cdfda06329e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:01.816512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae3a50-9c37-469c-a249-cdfda06329e4 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:01.817361 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 229/464] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:01.818508 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fc6b2-ec0d-4427-9042-8f5e2b93b75e None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:01.845441 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 236/465] 10.222.0.71 () {60 vars in 1132 bytes} [Thu Jul 21 20:26:01 2022] POST /identity/v3/users => generated 246 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:26:01.847198 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0507bd-6174-4361-9808-62c3d78cb010 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:01.847664 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0507bd-6174-4361-9808-62c3d78cb010 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:01.848128 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0507bd-6174-4361-9808-62c3d78cb010 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:01.849871 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 237/466] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:01.850894 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0507bd-6174-4361-9808-62c3d78cb010 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:01.913267 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:01.968622 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fc6b2-ec0d-4427-9042-8f5e2b93b75e None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:01.976936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0507bd-6174-4361-9808-62c3d78cb010 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:02.014784 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb687ed-5b2d-4087-83fc-bee566964721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:02.015474 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb687ed-5b2d-4087-83fc-bee566964721 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:02.015768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb687ed-5b2d-4087-83fc-bee566964721 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:02.016051 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb687ed-5b2d-4087-83fc-bee566964721 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:02.016861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb687ed-5b2d-4087-83fc-bee566964721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:02.017209 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb687ed-5b2d-4087-83fc-bee566964721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:02.019238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b53f39-f96b-4dc8-93a3-4405e1f43913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:02.019868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb687ed-5b2d-4087-83fc-bee566964721 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:02.020618 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b53f39-f96b-4dc8-93a3-4405e1f43913 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:02.021134 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b53f39-f96b-4dc8-93a3-4405e1f43913 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:02.021611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b53f39-f96b-4dc8-93a3-4405e1f43913 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:02.038375 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b53f39-f96b-4dc8-93a3-4405e1f43913 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:02.040242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 230/467] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:02.040539 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b53f39-f96b-4dc8-93a3-4405e1f43913 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:02.041841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd2df6-0a8c-4d07-af89-fd2b07ae8322 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:02.042152 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd2df6-0a8c-4d07-af89-fd2b07ae8322 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:02.042445 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd2df6-0a8c-4d07-af89-fd2b07ae8322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:02.064385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-f4b53f39-f96b-4dc8-93a3-4405e1f43913 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:02.064385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:02.064385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:02.064385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:02.064385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:02.064385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:02.064385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:02.064385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:02.064385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:02.064385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:02.064385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:02.064385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:02.064385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:02.064385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:02.064385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:02.065317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:02.065317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:02.065317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:02.065317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:02.065317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:02.065317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:02.065317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:02.065317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:02.065317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:02.065317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:02.065317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:02.065317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:02.065317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:02.065317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:02.065317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:02.065317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:02.065317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:02.066158 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:02.066158 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:02.066158 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:02.066158 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:02.066805 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 238/468] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:02.067993 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b53f39-f96b-4dc8-93a3-4405e1f43913 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:02.076558 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-b8bd2df6-0a8c-4d07-af89-fd2b07ae8322 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:02.078113 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-b8bd2df6-0a8c-4d07-af89-fd2b07ae8322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:02.110551 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:02.166223 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 231/469] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:02.167514 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd2df6-0a8c-4d07-af89-fd2b07ae8322 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:02.222101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b53f39-f96b-4dc8-93a3-4405e1f43913 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:02.224108 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:02.274535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d131f-068f-4e60-8c36-06f3e67f40f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:02.275207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d131f-068f-4e60-8c36-06f3e67f40f1 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:02.275493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d131f-068f-4e60-8c36-06f3e67f40f1 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:02.275772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d131f-068f-4e60-8c36-06f3e67f40f1 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:02.276485 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d131f-068f-4e60-8c36-06f3e67f40f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:02.279099 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d131f-068f-4e60-8c36-06f3e67f40f1 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:02.296691 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd2df6-0a8c-4d07-af89-fd2b07ae8322 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:02.317202 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 239/470] 10.222.0.71 () {60 vars in 1132 bytes} [Thu Jul 21 20:26:02 2022] POST /identity/v3/users => generated 248 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:26:02.317202 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d131f-068f-4e60-8c36-06f3e67f40f1 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:02.387242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af91ff9-84a4-476d-bbde-dfc950ed0d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:02.388757 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af91ff9-84a4-476d-bbde-dfc950ed0d77 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:02.389212 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af91ff9-84a4-476d-bbde-dfc950ed0d77 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:02.389635 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af91ff9-84a4-476d-bbde-dfc950ed0d77 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:02.390520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af91ff9-84a4-476d-bbde-dfc950ed0d77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:02.390983 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af91ff9-84a4-476d-bbde-dfc950ed0d77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:02.392505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af91ff9-84a4-476d-bbde-dfc950ed0d77 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:02.424409 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 232/471] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:02.430201 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af91ff9-84a4-476d-bbde-dfc950ed0d77 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:02.449098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d131f-068f-4e60-8c36-06f3e67f40f1 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:02.482920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:02.488271 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584526c-db6c-4b5c-8f07-46f0865271ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:02.489207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584526c-db6c-4b5c-8f07-46f0865271ec None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:02.489595 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584526c-db6c-4b5c-8f07-46f0865271ec None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:02.489943 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584526c-db6c-4b5c-8f07-46f0865271ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:02.502849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584526c-db6c-4b5c-8f07-46f0865271ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:02.504517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584526c-db6c-4b5c-8f07-46f0865271ec None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:02.523745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-e584526c-db6c-4b5c-8f07-46f0865271ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:02.523745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:02.523745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:02.523745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:02.523745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:02.523745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:02.523745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:02.523745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:02.523745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:02.523745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:02.523745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:02.523745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:02.523745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:02.523745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:02.523745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:02.524658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:02.524658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:02.524658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:02.524658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:02.524658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:02.524658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:02.524658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:02.524658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:02.524658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:02.524658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:02.524658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:02.524658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:02.524658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:02.524658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:02.524658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:02.524658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:02.524658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:02.525552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:02.525552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:02.525552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:02.525552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:02.525552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 240/472] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:02.526741 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e7937-c966-4af6-95fe-c31b24918f1d None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:02.527091 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e7937-c966-4af6-95fe-c31b24918f1d None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:02.527434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e7937-c966-4af6-95fe-c31b24918f1d None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:02.529212 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 241/473] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:02.530585 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e7937-c966-4af6-95fe-c31b24918f1d None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:02.545609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:02.592711 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af91ff9-84a4-476d-bbde-dfc950ed0d77 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:02.648341 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794046cd-2746-4dbd-8a05-48a3d55c0d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:02.649139 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794046cd-2746-4dbd-8a05-48a3d55c0d54 None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:02.649493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794046cd-2746-4dbd-8a05-48a3d55c0d54 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:02.649768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794046cd-2746-4dbd-8a05-48a3d55c0d54 None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:02.650474 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794046cd-2746-4dbd-8a05-48a3d55c0d54 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:02.651914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794046cd-2746-4dbd-8a05-48a3d55c0d54 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:02.676382 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 233/474] 10.222.0.71 () {60 vars in 1132 bytes} [Thu Jul 21 20:26:02 2022] POST /identity/v3/users => generated 248 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:26:02.679985 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71737af-d8d4-4a0b-bce5-3470ecf2a4a1 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:02.680272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71737af-d8d4-4a0b-bce5-3470ecf2a4a1 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:02.680551 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71737af-d8d4-4a0b-bce5-3470ecf2a4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:02.682635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e7937-c966-4af6-95fe-c31b24918f1d None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:02.710600 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-a71737af-d8d4-4a0b-bce5-3470ecf2a4a1 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:02.711858 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-a71737af-d8d4-4a0b-bce5-3470ecf2a4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:02.768876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ea507-c5dc-4775-b01c-6a435768531a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:02.770084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ea507-c5dc-4775-b01c-6a435768531a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:02.770369 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ea507-c5dc-4775-b01c-6a435768531a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:02.771247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ea507-c5dc-4775-b01c-6a435768531a None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:02.774433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ea507-c5dc-4775-b01c-6a435768531a None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:02.774767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ea507-c5dc-4775-b01c-6a435768531a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:02.778197 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ea507-c5dc-4775-b01c-6a435768531a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:02.813242 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 242/475] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:02.818734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ea507-c5dc-4775-b01c-6a435768531a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:02.843904 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 234/476] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:02.883365 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71737af-d8d4-4a0b-bce5-3470ecf2a4a1 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:02.895941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:02.949064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:02.997745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ea507-c5dc-4775-b01c-6a435768531a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:03.042688 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41477c5e-dbc8-4175-a0cf-11e8efbc1280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:03.043377 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41477c5e-dbc8-4175-a0cf-11e8efbc1280 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:03.043575 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71737af-d8d4-4a0b-bce5-3470ecf2a4a1 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:03.044002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41477c5e-dbc8-4175-a0cf-11e8efbc1280 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:03.044002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41477c5e-dbc8-4175-a0cf-11e8efbc1280 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:03.044718 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41477c5e-dbc8-4175-a0cf-11e8efbc1280 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:03.047566 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41477c5e-dbc8-4175-a0cf-11e8efbc1280 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:03.076014 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 243/477] 10.222.0.71 () {60 vars in 1132 bytes} [Thu Jul 21 20:26:02 2022] POST /identity/v3/users => generated 249 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:26:03.097002 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7a069-e157-49fc-a622-63e2811b6d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:03.097668 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7a069-e157-49fc-a622-63e2811b6d08 None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:03.097950 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7a069-e157-49fc-a622-63e2811b6d08 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:03.098228 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7a069-e157-49fc-a622-63e2811b6d08 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:03.099013 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7a069-e157-49fc-a622-63e2811b6d08 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:03.100726 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7a069-e157-49fc-a622-63e2811b6d08 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:03.109619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.sql.core [None req-cfe7a069-e157-49fc-a622-63e2811b6d08 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 20:26:03.109619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 20:26:03.109619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [parameters: {'id': '44f1c64eda1a486ba3cd87b7971a9541', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 20:26:03.109619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=103664) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 20:26:03.127345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-cfe7a069-e157-49fc-a622-63e2811b6d08 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 20:26:03.127345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:03.127345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 20:26:03.127345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 20:26:03.127345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 20:26:03.127345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 20:26:03.127345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 20:26:03.127345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 20:26:03.127345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 20:26:03.127345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application conn.query(q) Jul 21 20:26:03.127345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 20:26:03.127345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 20:26:03.127345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 20:26:03.127345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application result.read() Jul 21 20:26:03.127345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 20:26:03.128282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 20:26:03.128282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 20:26:03.128282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 20:26:03.128282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 20:26:03.128282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 20:26:03.128282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 20:26:03.128282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 20:26:03.128282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 20:26:03.128282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:03.128282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 20:26:03.128282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:03.128282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:03.128282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 20:26:03.128282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 20:26:03.128282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 21 20:26:03.128282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return ref.to_dict() Jul 21 20:26:03.128282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 20:26:03.128282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application next(self.gen) Jul 21 20:26:03.129248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 20:26:03.129248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application yield resource Jul 21 20:26:03.129248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 20:26:03.129248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application next(self.gen) Jul 21 20:26:03.129248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 20:26:03.129248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 20:26:03.129248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 20:26:03.129248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 20:26:03.129248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 20:26:03.129248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise self.value Jul 21 20:26:03.129248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 20:26:03.129248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 20:26:03.129248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 20:26:03.129248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application session.commit() Jul 21 20:26:03.129248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 20:26:03.129248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 20:26:03.129248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 20:26:03.130148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 20:26:03.130148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 20:26:03.130148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self.session.flush() Jul 21 20:26:03.130148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 20:26:03.130148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 20:26:03.130148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 20:26:03.130148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 20:26:03.130148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 20:26:03.130148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application compat.raise_( Jul 21 20:26:03.130148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 20:26:03.130148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception Jul 21 20:26:03.130148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 20:26:03.130148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 20:26:03.130148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 20:26:03.130148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 20:26:03.130148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 20:26:03.130148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 20:26:03.131091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 20:26:03.131091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 20:26:03.131091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 20:26:03.131091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 20:26:03.131091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 20:26:03.131091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 20:26:03.131091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 20:26:03.131091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 20:26:03.131091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 20:26:03.131091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 20:26:03.131091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 20:26:03.131091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 20:26:03.131091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 20:26:03.131091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 20:26:03.131091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 20:26:03.131091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception Jul 21 20:26:03.131933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 20:26:03.131933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 20:26:03.131933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 20:26:03.131933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 20:26:03.131933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 20:26:03.131933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 20:26:03.131933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 20:26:03.131933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application conn.query(q) Jul 21 20:26:03.131933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 20:26:03.131933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 20:26:03.131933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 20:26:03.131933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application result.read() Jul 21 20:26:03.131933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 20:26:03.131933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 20:26:03.131933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 20:26:03.131933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 20:26:03.132741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 20:26:03.132741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 20:26:03.132741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 20:26:03.132741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 20:26:03.132741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 20:26:03.132741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 20:26:03.132741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [parameters: {'id': '44f1c64eda1a486ba3cd87b7971a9541', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 20:26:03.132741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 20:26:03.132741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:03.132741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:03.132741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:03.132741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:03.132741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:03.132741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:03.132741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:03.132741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:03.135466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:03.135466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:03.135466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:03.135466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:03.135466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:03.135466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:03.135466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 21 20:26:03.135466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 21 20:26:03.135466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:03.135466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:03.135466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 21 20:26:03.135466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 21 20:26:03.135466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 20:26:03.135466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 20:26:03.135466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 20:26:03.135466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:03.137552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 235/478] 10.222.0.71 () {60 vars in 1131 bytes} [Thu Jul 21 20:26:02 2022] POST /identity/v3/roles => generated 144 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:26:03.137690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41477c5e-dbc8-4175-a0cf-11e8efbc1280 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:03.299228 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41477c5e-dbc8-4175-a0cf-11e8efbc1280 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:03.357775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:03.368476 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfef60-5342-4dcd-92ce-89cf46138ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:03.369331 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfef60-5342-4dcd-92ce-89cf46138ee1 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:03.369612 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfef60-5342-4dcd-92ce-89cf46138ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:03.369886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfef60-5342-4dcd-92ce-89cf46138ee1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:03.390137 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfef60-5342-4dcd-92ce-89cf46138ee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:03.398325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-f8dfef60-5342-4dcd-92ce-89cf46138ee1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:26:03.398325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:03.398325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:03.398325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:03.398325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:03.398325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:03.398325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:03.398325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:03.398325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:03.398325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:03.398325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:03.398325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:03.398325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:03.398325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:03.398325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:03.398325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:03.400534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:03.400534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:03.400534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:03.400534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:03.400534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:03.400534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:03.400534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:03.400534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:03.400534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:03.400534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:03.400534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:03.400534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:03.400534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:03.400534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:03.400534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:03.400534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:03.402663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:03.402663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:03.402663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:03.402663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:03.402663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:03.402663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:03.402663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:03.402663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:03.402663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:03.402663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:03.402663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:26:03.402663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:03.402663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 244/479] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:26:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:03.410887 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7a069-e157-49fc-a622-63e2811b6d08 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:03.524162 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:03.567810 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7a069-e157-49fc-a622-63e2811b6d08 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:03.613526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23565fc6-f98c-4136-b7db-33674d0ad3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:03.614124 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23565fc6-f98c-4136-b7db-33674d0ad3db None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:03.614497 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23565fc6-f98c-4136-b7db-33674d0ad3db None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:03.614858 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23565fc6-f98c-4136-b7db-33674d0ad3db None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:03.615695 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23565fc6-f98c-4136-b7db-33674d0ad3db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:03.616002 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23565fc6-f98c-4136-b7db-33674d0ad3db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:03.617413 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23565fc6-f98c-4136-b7db-33674d0ad3db None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:03.627147 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 236/480] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:04.294922 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720aba2-9bad-4bd1-91df-80b6bc35bca6 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:04.294922 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720aba2-9bad-4bd1-91df-80b6bc35bca6 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:04.294922 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720aba2-9bad-4bd1-91df-80b6bc35bca6 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:04.296165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 245/481] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:04.302012 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac5a44-c436-4d36-a912-7f98de0c6c57 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:04.302879 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac5a44-c436-4d36-a912-7f98de0c6c57 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:04.303393 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac5a44-c436-4d36-a912-7f98de0c6c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:04.334360 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-c1ac5a44-c436-4d36-a912-7f98de0c6c57 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:04.336010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c1ac5a44-c436-4d36-a912-7f98de0c6c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:04.440091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 237/482] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:26:04.476906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720aba2-9bad-4bd1-91df-80b6bc35bca6 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:04.641849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720aba2-9bad-4bd1-91df-80b6bc35bca6 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:04.688371 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b8aaa-4fe9-4d00-91ae-4d3a5ba56b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:04.689581 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b8aaa-4fe9-4d00-91ae-4d3a5ba56b02 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:04.690157 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b8aaa-4fe9-4d00-91ae-4d3a5ba56b02 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:04.690640 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b8aaa-4fe9-4d00-91ae-4d3a5ba56b02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:04.703409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b8aaa-4fe9-4d00-91ae-4d3a5ba56b02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:04.711921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-465b8aaa-4fe9-4d00-91ae-4d3a5ba56b02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:04.711921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:04.711921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:04.711921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:04.711921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:04.711921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:04.711921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:04.711921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:04.711921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:04.711921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:04.711921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:04.711921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:04.711921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:04.711921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:04.711921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:04.711921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:04.712840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:04.712840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:04.712840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:04.712840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:04.712840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:04.712840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:04.712840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:04.712840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:04.712840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:04.712840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:04.712840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:04.712840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:04.712840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:04.712840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:04.712840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:04.712840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:04.713706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:04.713706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:04.713706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:04.713706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:04.713706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:04.713706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:04.713706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:04.713706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:04.713706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:04.713706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:04.713706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:04.713706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:04.716507 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 246/483] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:04.730230 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac5a44-c436-4d36-a912-7f98de0c6c57 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:04.821546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:04.868573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac5a44-c436-4d36-a912-7f98de0c6c57 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:04.883095 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b54360-cd44-48d8-a981-1d7f7682f55e None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:04.883398 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b54360-cd44-48d8-a981-1d7f7682f55e None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:04.883730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b54360-cd44-48d8-a981-1d7f7682f55e None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:04.885401 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 247/484] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:26:04.891168 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4300e0b-7086-402e-9224-059fd5a42fd2 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:04.891446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4300e0b-7086-402e-9224-059fd5a42fd2 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:04.891729 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4300e0b-7086-402e-9224-059fd5a42fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:04.916383 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-f4300e0b-7086-402e-9224-059fd5a42fd2 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:04.917636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f4300e0b-7086-402e-9224-059fd5a42fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:04.931792 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affaccd-fc9d-4cef-ab54-7b7fb956929b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:04.932479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affaccd-fc9d-4cef-ab54-7b7fb956929b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:04.932759 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affaccd-fc9d-4cef-ab54-7b7fb956929b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:04.933034 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affaccd-fc9d-4cef-ab54-7b7fb956929b None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:04.933880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affaccd-fc9d-4cef-ab54-7b7fb956929b None admin] RBAC: Adding query filter params (name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:04.934189 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affaccd-fc9d-4cef-ab54-7b7fb956929b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:04.937814 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affaccd-fc9d-4cef-ab54-7b7fb956929b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:04.952127 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 238/485] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:26:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:04.957356 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affaccd-fc9d-4cef-ab54-7b7fb956929b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:05.006284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:05.063458 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 248/486] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:26:05.118678 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4300e0b-7086-402e-9224-059fd5a42fd2 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:05.157150 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affaccd-fc9d-4cef-ab54-7b7fb956929b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:05.205606 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:05.230087 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843221f7-1267-436f-821e-ad1700d23e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:05.230906 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843221f7-1267-436f-821e-ad1700d23e2f None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:05.231189 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843221f7-1267-436f-821e-ad1700d23e2f None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:05.231463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843221f7-1267-436f-821e-ad1700d23e2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:05.254086 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843221f7-1267-436f-821e-ad1700d23e2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:05.255607 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843221f7-1267-436f-821e-ad1700d23e2f None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:05.290314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-843221f7-1267-436f-821e-ad1700d23e2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:05.290314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:05.290314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:05.290314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:05.290314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:05.290314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:05.290314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:05.290314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:05.290314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:05.290314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:05.290314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:05.290314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:05.290314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:05.290314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:05.290314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:05.291265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:05.291265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:05.291265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:05.291265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:05.291265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:05.291265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:05.291265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:05.291265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:05.291265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:05.291265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:05.291265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:05.291265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:05.291265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:05.291265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:05.291265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:05.291265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:05.291265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:05.292161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:05.292161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:05.292161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:05.292161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:05.292161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 239/487] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:05.300524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843221f7-1267-436f-821e-ad1700d23e2f None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:05.307228 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4300e0b-7086-402e-9224-059fd5a42fd2 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:05.365638 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e62d23-6211-4efe-8870-54057e216d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:05.366639 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e62d23-6211-4efe-8870-54057e216d1a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:05.366923 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e62d23-6211-4efe-8870-54057e216d1a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:05.367199 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e62d23-6211-4efe-8870-54057e216d1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:05.382218 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e62d23-6211-4efe-8870-54057e216d1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:05.391948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-19e62d23-6211-4efe-8870-54057e216d1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:05.391948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:05.391948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:05.391948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:05.391948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:05.391948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:05.391948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:05.391948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:05.391948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:05.391948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:05.391948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:05.391948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:05.391948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:05.391948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:05.391948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:05.391948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:05.393025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:05.393025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:05.393025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:05.393025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:05.393025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:05.393025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:05.393025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:05.393025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:05.393025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:05.393025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:05.393025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:05.393025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:05.393025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:05.393025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:05.393025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:05.393025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:05.394172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:05.394172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:05.394172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:05.394172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:05.394172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:05.394172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:05.394172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:05.394172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:05.394172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:05.394172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:05.394172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:05.394172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:05.394915 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 249/488] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:26:05.400432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a3036-d6b0-42bf-a147-38c26fcfb438 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:05.400432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a3036-d6b0-42bf-a147-38c26fcfb438 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:05.400432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a3036-d6b0-42bf-a147-38c26fcfb438 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:05.400432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 250/489] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:05.402076 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a3036-d6b0-42bf-a147-38c26fcfb438 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:05.455582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:05.456904 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843221f7-1267-436f-821e-ad1700d23e2f None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:05.501469 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62508c19-9fb3-4afc-b263-b3252084816d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:05.502483 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62508c19-9fb3-4afc-b263-b3252084816d None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:05.502997 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62508c19-9fb3-4afc-b263-b3252084816d None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:05.503500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62508c19-9fb3-4afc-b263-b3252084816d None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:05.504544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62508c19-9fb3-4afc-b263-b3252084816d None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:05.505061 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62508c19-9fb3-4afc-b263-b3252084816d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:05.506710 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62508c19-9fb3-4afc-b263-b3252084816d None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:05.522540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 240/490] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:05.524258 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7575733-2208-42c7-8e43-e03760c0ae83 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:05.524861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7575733-2208-42c7-8e43-e03760c0ae83 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:05.525362 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7575733-2208-42c7-8e43-e03760c0ae83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:05.546148 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a3036-d6b0-42bf-a147-38c26fcfb438 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:05.551042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-f7575733-2208-42c7-8e43-e03760c0ae83 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:05.552326 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-f7575733-2208-42c7-8e43-e03760c0ae83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:05.584202 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b04a1-66d5-448b-9f80-2d5e4fa6f64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:05.585270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b04a1-66d5-448b-9f80-2d5e4fa6f64e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:05.585762 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b04a1-66d5-448b-9f80-2d5e4fa6f64e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:05.586265 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b04a1-66d5-448b-9f80-2d5e4fa6f64e None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:05.587217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b04a1-66d5-448b-9f80-2d5e4fa6f64e None admin] RBAC: Adding query filter params (name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:05.587740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b04a1-66d5-448b-9f80-2d5e4fa6f64e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:05.589374 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b04a1-66d5-448b-9f80-2d5e4fa6f64e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:05.592719 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:05.600860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 251/491] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:26:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:05.602203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b04a1-66d5-448b-9f80-2d5e4fa6f64e None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:05.649293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 241/492] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:05.650531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7575733-2208-42c7-8e43-e03760c0ae83 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:05.738392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:05.742925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b04a1-66d5-448b-9f80-2d5e4fa6f64e None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:05.787987 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022061cf-34e4-4aa6-a91b-9dd4484c6936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:05.789142 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022061cf-34e4-4aa6-a91b-9dd4484c6936 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:05.789527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022061cf-34e4-4aa6-a91b-9dd4484c6936 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:05.790267 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022061cf-34e4-4aa6-a91b-9dd4484c6936 None admin] PATH_INFO: `/v3/users/nova` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:05.797356 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7575733-2208-42c7-8e43-e03760c0ae83 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:05.802904 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022061cf-34e4-4aa6-a91b-9dd4484c6936 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:05.804769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022061cf-34e4-4aa6-a91b-9dd4484c6936 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:05.829100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-022061cf-34e4-4aa6-a91b-9dd4484c6936 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:26:05.829100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:05.829100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:05.829100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:05.829100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:05.829100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:05.829100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:05.829100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:05.829100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:05.829100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:05.829100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:05.829100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:05.829100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:05.829100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:05.829100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:05.829100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:05.830220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:05.830220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:05.830220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:05.830220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:05.830220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:05.830220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:05.830220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:05.830220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:05.830220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:05.830220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:05.830220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:05.830220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:05.830220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:05.830220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:05.830220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:05.830220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:05.830220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:05.830220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:05.831140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:05.831140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:05.831140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:05.831140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:05.831140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:05.831140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:05.831140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:05.831140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:05.831140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:26:05.831140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:05.833156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 252/493] 10.222.0.71 () {56 vars in 1093 bytes} [Thu Jul 21 20:26:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:05.834300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022061cf-34e4-4aa6-a91b-9dd4484c6936 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:05.851035 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929543d-9e68-4ca6-930f-6669d9b5d1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:05.851888 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929543d-9e68-4ca6-930f-6669d9b5d1b2 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:05.852181 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929543d-9e68-4ca6-930f-6669d9b5d1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:05.852943 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929543d-9e68-4ca6-930f-6669d9b5d1b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:05.868296 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929543d-9e68-4ca6-930f-6669d9b5d1b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:05.870061 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929543d-9e68-4ca6-930f-6669d9b5d1b2 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:05.894931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-2929543d-9e68-4ca6-930f-6669d9b5d1b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:05.894931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:05.894931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:05.894931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:05.894931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:05.894931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:05.894931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:05.894931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:05.894931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:05.894931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:05.894931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:05.894931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:05.894931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:05.894931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:05.894931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:05.895869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:05.895869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:05.895869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:05.895869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:05.895869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:05.895869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:05.895869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:05.895869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:05.895869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:05.895869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:05.895869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:05.895869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:05.895869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:05.895869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:05.895869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:05.895869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:05.895869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:05.896744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:05.896744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:05.896744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:05.896744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:05.901320 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 242/494] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:05.902256 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929543d-9e68-4ca6-930f-6669d9b5d1b2 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:05.930199 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:05.967177 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022061cf-34e4-4aa6-a91b-9dd4484c6936 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:06.009680 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b91768-d2db-46a5-bdfa-bbab3498b670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:06.010829 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b91768-d2db-46a5-bdfa-bbab3498b670 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:06.011318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b91768-d2db-46a5-bdfa-bbab3498b670 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:06.011794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b91768-d2db-46a5-bdfa-bbab3498b670 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:06.023579 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b91768-d2db-46a5-bdfa-bbab3498b670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:06.032333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-03b91768-d2db-46a5-bdfa-bbab3498b670 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:06.032333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:06.032333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:06.032333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:06.032333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:06.032333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:06.032333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:06.032333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:06.032333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:06.032333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:06.032333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:06.032333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:06.032333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:06.032333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:06.032333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:06.032333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:06.033326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:06.033326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:06.033326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:06.033326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:06.033326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:06.033326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:06.033326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:06.033326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:06.033326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:06.033326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:06.033326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:06.033326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:06.033326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:06.033326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:06.033326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:06.033326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:06.034129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:06.034129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:06.034129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:06.034129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:06.034129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:06.034129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:06.034129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:06.034129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:06.034129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:06.034129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:06.034129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:06.034129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:06.036726 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 253/495] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:06.037763 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b91768-d2db-46a5-bdfa-bbab3498b670 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:06.038791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929543d-9e68-4ca6-930f-6669d9b5d1b2 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:06.060017 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:06.095788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360b47e-afe2-46fd-aa7c-0623d9683e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:06.096757 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360b47e-afe2-46fd-aa7c-0623d9683e74 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:06.097677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360b47e-afe2-46fd-aa7c-0623d9683e74 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:06.098196 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360b47e-afe2-46fd-aa7c-0623d9683e74 None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:06.101199 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360b47e-afe2-46fd-aa7c-0623d9683e74 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:06.101768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360b47e-afe2-46fd-aa7c-0623d9683e74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:06.105408 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360b47e-afe2-46fd-aa7c-0623d9683e74 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:06.170441 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b91768-d2db-46a5-bdfa-bbab3498b670 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:06.183580 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 243/496] 10.222.0.71 () {56 vars in 1133 bytes} [Thu Jul 21 20:26:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:06.185520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360b47e-afe2-46fd-aa7c-0623d9683e74 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:06.221092 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c37b9-4b1c-4238-8559-8147dd1cf63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:06.221779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c37b9-4b1c-4238-8559-8147dd1cf63f None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:06.222057 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c37b9-4b1c-4238-8559-8147dd1cf63f None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:06.222334 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c37b9-4b1c-4238-8559-8147dd1cf63f None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:06.223076 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c37b9-4b1c-4238-8559-8147dd1cf63f None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:06.223383 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c37b9-4b1c-4238-8559-8147dd1cf63f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:06.225062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c37b9-4b1c-4238-8559-8147dd1cf63f None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:06.237930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 254/497] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:06.238834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c37b9-4b1c-4238-8559-8147dd1cf63f None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:06.302193 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:06.330258 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360b47e-afe2-46fd-aa7c-0623d9683e74 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:06.339660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:06.367041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085cd1a-5edd-47e2-a9f0-c0854fc678c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:06.367766 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085cd1a-5edd-47e2-a9f0-c0854fc678c0 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:06.368020 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085cd1a-5edd-47e2-a9f0-c0854fc678c0 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:06.368322 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085cd1a-5edd-47e2-a9f0-c0854fc678c0 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:06.368565 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c37b9-4b1c-4238-8559-8147dd1cf63f None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:06.369095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1085cd1a-5edd-47e2-a9f0-c0854fc678c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:06.369407 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1085cd1a-5edd-47e2-a9f0-c0854fc678c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:06.370825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1085cd1a-5edd-47e2-a9f0-c0854fc678c0 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:06.381145 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 244/498] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:26:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:06.383074 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dac6b8-2990-401d-9300-67cea2cb8e9f None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:06.383368 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dac6b8-2990-401d-9300-67cea2cb8e9f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:06.383645 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dac6b8-2990-401d-9300-67cea2cb8e9f None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:06.384971 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 245/499] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:06.386101 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e76552-c570-42da-a0c9-126a1e143bc0 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:06.386388 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e76552-c570-42da-a0c9-126a1e143bc0 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:06.386655 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e76552-c570-42da-a0c9-126a1e143bc0 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:06.387902 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 246/500] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:06.388619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e76552-c570-42da-a0c9-126a1e143bc0 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:06.411123 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86442804-44aa-4da9-b490-9e2eb8a90d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:06.411772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86442804-44aa-4da9-b490-9e2eb8a90d6c None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:06.412054 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86442804-44aa-4da9-b490-9e2eb8a90d6c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:06.412335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86442804-44aa-4da9-b490-9e2eb8a90d6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:06.494285 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86442804-44aa-4da9-b490-9e2eb8a90d6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:06.495817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86442804-44aa-4da9-b490-9e2eb8a90d6c None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:06.512335 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e76552-c570-42da-a0c9-126a1e143bc0 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:06.548296 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7301b45-ff81-49b7-9aa1-a4f768aba83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:06.549113 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7301b45-ff81-49b7-9aa1-a4f768aba83d None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:06.549414 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7301b45-ff81-49b7-9aa1-a4f768aba83d None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:06.549661 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7301b45-ff81-49b7-9aa1-a4f768aba83d None admin] PATH_INFO: `/v3/users/glance` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:06.557592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7301b45-ff81-49b7-9aa1-a4f768aba83d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:06.558983 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7301b45-ff81-49b7-9aa1-a4f768aba83d None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:06.560883 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:06.576941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-a7301b45-ff81-49b7-9aa1-a4f768aba83d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 20:26:06.576941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:06.576941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:06.576941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:06.576941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:06.576941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:06.576941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:06.576941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:06.576941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:06.576941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:06.576941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:06.576941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:06.576941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:06.576941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:06.576941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:06.576941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:06.578133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:06.578133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:06.578133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:06.578133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:06.578133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:06.578133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:06.578133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:06.578133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:06.578133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:06.578133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:06.578133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:06.578133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:06.578133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:06.578133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:06.578133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:06.578133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:06.578133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:06.578133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:06.579205 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:06.579205 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:06.579205 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:06.579205 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:06.579205 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:06.579205 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:06.579205 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:06.579205 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:06.579205 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 20:26:06.579205 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:06.579205 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 247/501] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:26:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:06.579205 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7301b45-ff81-49b7-9aa1-a4f768aba83d None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:06.584100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:06.609371 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 255/502] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:26:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:26:06.610833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4802923-e3a4-4b00-bd7a-6e69a6075d47 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:06.610950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4802923-e3a4-4b00-bd7a-6e69a6075d47 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:06.611238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4802923-e3a4-4b00-bd7a-6e69a6075d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:06.629693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-a4802923-e3a4-4b00-bd7a-6e69a6075d47 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:06.630911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a4802923-e3a4-4b00-bd7a-6e69a6075d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:06.684482 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7301b45-ff81-49b7-9aa1-a4f768aba83d None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:06.699196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 256/503] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:06.700674 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1e101-c418-4605-a13d-8f05e03abcda None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:06.700959 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1e101-c418-4605-a13d-8f05e03abcda None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:06.701272 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1e101-c418-4605-a13d-8f05e03abcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:06.717759 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-fde1e101-c418-4605-a13d-8f05e03abcda None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:06.718955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-fde1e101-c418-4605-a13d-8f05e03abcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:06.730561 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770358f-623c-4c7f-9c3f-21947ee0824d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:06.731366 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770358f-623c-4c7f-9c3f-21947ee0824d None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:06.732239 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770358f-623c-4c7f-9c3f-21947ee0824d None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:06.732502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770358f-623c-4c7f-9c3f-21947ee0824d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:06.743346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770358f-623c-4c7f-9c3f-21947ee0824d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:06.744824 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770358f-623c-4c7f-9c3f-21947ee0824d None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:06.762266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-5770358f-623c-4c7f-9c3f-21947ee0824d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:06.762266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:06.762266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:06.762266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:06.762266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:06.762266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:06.762266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:06.762266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:06.762266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:06.762266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:06.762266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:06.762266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:06.762266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:06.762266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:06.762266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:06.763235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:06.763235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:06.763235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:06.763235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:06.763235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:06.763235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:06.763235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:06.763235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:06.763235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:06.763235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:06.763235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:06.763235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:06.763235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:06.763235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:06.763235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:06.763235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:06.763235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:06.764096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:06.764096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:06.764096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:06.764096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:06.764096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 248/504] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:06.764373 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770358f-623c-4c7f-9c3f-21947ee0824d None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:06.789402 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 257/505] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:06.790253 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1e101-c418-4605-a13d-8f05e03abcda None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:06.828010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:06.832083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:06.880091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770358f-623c-4c7f-9c3f-21947ee0824d None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:06.894627 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1e101-c418-4605-a13d-8f05e03abcda None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:06.915412 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e045ad9-adce-43a7-8ad2-f4960ba8be23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:06.916071 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e045ad9-adce-43a7-8ad2-f4960ba8be23 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:06.916350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e045ad9-adce-43a7-8ad2-f4960ba8be23 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:06.916621 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e045ad9-adce-43a7-8ad2-f4960ba8be23 None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:06.917512 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e045ad9-adce-43a7-8ad2-f4960ba8be23 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:06.917817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e045ad9-adce-43a7-8ad2-f4960ba8be23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:06.919808 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e045ad9-adce-43a7-8ad2-f4960ba8be23 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:06.928703 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75866d7c-3713-4cb0-9f31-ae40ea61baba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:06.929528 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75866d7c-3713-4cb0-9f31-ae40ea61baba None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:06.929811 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75866d7c-3713-4cb0-9f31-ae40ea61baba None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:06.930102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75866d7c-3713-4cb0-9f31-ae40ea61baba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:06.937859 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 249/506] 10.222.0.71 () {56 vars in 1137 bytes} [Thu Jul 21 20:26:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:06.938671 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e045ad9-adce-43a7-8ad2-f4960ba8be23 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:06.946600 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75866d7c-3713-4cb0-9f31-ae40ea61baba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:06.948923 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75866d7c-3713-4cb0-9f31-ae40ea61baba None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:06.969552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-75866d7c-3713-4cb0-9f31-ae40ea61baba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:06.969552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:06.969552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:06.969552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:06.969552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:06.969552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:06.969552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:06.969552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:06.969552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:06.969552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:06.969552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:06.969552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:06.969552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:06.969552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:06.969552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:06.970510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:06.970510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:06.970510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:06.970510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:06.970510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:06.970510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:06.970510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:06.970510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:06.970510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:06.970510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:06.970510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:06.970510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:06.970510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:06.970510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:06.970510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:06.970510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:06.970510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:06.971357 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:06.971357 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:06.971357 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:06.971357 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:06.971357 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 258/507] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:06.971691 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75866d7c-3713-4cb0-9f31-ae40ea61baba None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:07.046905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e045ad9-adce-43a7-8ad2-f4960ba8be23 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:07.056187 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:07.063155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:07.074189 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75866d7c-3713-4cb0-9f31-ae40ea61baba None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:07.085003 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea9165-df92-40ff-9b18-f5bdfdb826d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:07.085764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea9165-df92-40ff-9b18-f5bdfdb826d4 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:07.086045 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea9165-df92-40ff-9b18-f5bdfdb826d4 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:07.086317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea9165-df92-40ff-9b18-f5bdfdb826d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:07.097920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea9165-df92-40ff-9b18-f5bdfdb826d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:07.099433 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea9165-df92-40ff-9b18-f5bdfdb826d4 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:07.112367 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f20ab-ea8c-4274-b0cf-963db8569fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:07.113032 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f20ab-ea8c-4274-b0cf-963db8569fa9 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:07.113344 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f20ab-ea8c-4274-b0cf-963db8569fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:07.113620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f20ab-ea8c-4274-b0cf-963db8569fa9 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:07.114362 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f20ab-ea8c-4274-b0cf-963db8569fa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:07.114669 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f20ab-ea8c-4274-b0cf-963db8569fa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:07.116063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f20ab-ea8c-4274-b0cf-963db8569fa9 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:07.119022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-61ea9165-df92-40ff-9b18-f5bdfdb826d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:07.119022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:07.119022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:07.119022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:07.119022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:07.119022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:07.119022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:07.119022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:07.119022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:07.119022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:07.119022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:07.119022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:07.119022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:07.119022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:07.119022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:07.120936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:07.120936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:07.120936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:07.120936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:07.120936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:07.120936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:07.120936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:07.120936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:07.120936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:07.120936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:07.120936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:07.120936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:07.120936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:07.120936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:07.120936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:07.120936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:07.120936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:07.123668 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:07.123668 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:07.123668 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:07.123668 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:07.123668 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 250/508] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:07.123668 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea9165-df92-40ff-9b18-f5bdfdb826d4 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:07.134240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 259/509] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:07.135090 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f20ab-ea8c-4274-b0cf-963db8569fa9 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:07.236320 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea9165-df92-40ff-9b18-f5bdfdb826d4 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:07.237108 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f20ab-ea8c-4274-b0cf-963db8569fa9 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:07.270098 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be9b41-9f9b-4c74-a0b1-6552a4420adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:07.270885 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be9b41-9f9b-4c74-a0b1-6552a4420adb None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:07.271166 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be9b41-9f9b-4c74-a0b1-6552a4420adb None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:07.271438 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be9b41-9f9b-4c74-a0b1-6552a4420adb None admin] PATH_INFO: `/v3/roles/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:07.272287 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6732f-afd3-448d-9045-b2f8a8cf3e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:07.272931 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6732f-afd3-448d-9045-b2f8a8cf3e5d None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:07.273253 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6732f-afd3-448d-9045-b2f8a8cf3e5d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:07.273532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6732f-afd3-448d-9045-b2f8a8cf3e5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:07.282683 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be9b41-9f9b-4c74-a0b1-6552a4420adb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:07.291291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-51be9b41-9f9b-4c74-a0b1-6552a4420adb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:07.291291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:07.291291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:07.291291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:07.291291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:07.291291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:07.291291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:07.291291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:07.291291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:07.291291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:07.291291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:07.291291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:07.291291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:07.291291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:07.291291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:07.291291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:07.292240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:07.292240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:07.292240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:07.292240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:07.292240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:07.292240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:07.292240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:07.292240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:07.292240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:07.292240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:07.292240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:07.292240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:07.292240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:07.292240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:07.292240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:07.292240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:07.293042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:07.293042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:07.293042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:07.293042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:07.293042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:07.293042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:07.293042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:07.293042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:07.293042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:07.293042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:07.293042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:07.293042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:07.293042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 251/510] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:07.293769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be9b41-9f9b-4c74-a0b1-6552a4420adb None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:07.299091 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:07.318748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:07.355464 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6732f-afd3-448d-9045-b2f8a8cf3e5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:07.356873 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6732f-afd3-448d-9045-b2f8a8cf3e5d None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:07.401922 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be9b41-9f9b-4c74-a0b1-6552a4420adb None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:07.434418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca630ac8-91cf-483f-b29a-a79241d3699e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:07.435623 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca630ac8-91cf-483f-b29a-a79241d3699e None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:07.435910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca630ac8-91cf-483f-b29a-a79241d3699e None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:07.436193 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca630ac8-91cf-483f-b29a-a79241d3699e None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:07.436920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca630ac8-91cf-483f-b29a-a79241d3699e None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:07.437267 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca630ac8-91cf-483f-b29a-a79241d3699e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:07.438656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca630ac8-91cf-483f-b29a-a79241d3699e None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:07.450193 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 252/511] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:07.451569 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca630ac8-91cf-483f-b29a-a79241d3699e None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:07.460942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 260/512] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:26:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:26:07.461979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6732f-afd3-448d-9045-b2f8a8cf3e5d None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:07.519587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:07.541820 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:07.563711 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca630ac8-91cf-483f-b29a-a79241d3699e None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:07.576689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6732f-afd3-448d-9045-b2f8a8cf3e5d None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:07.602602 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33fec9-376e-4a81-87fd-db0f549aa0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:07.603279 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33fec9-376e-4a81-87fd-db0f549aa0b3 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:07.603562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33fec9-376e-4a81-87fd-db0f549aa0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:07.603849 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33fec9-376e-4a81-87fd-db0f549aa0b3 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:07.604589 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33fec9-376e-4a81-87fd-db0f549aa0b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:07.604898 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33fec9-376e-4a81-87fd-db0f549aa0b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:07.606362 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33fec9-376e-4a81-87fd-db0f549aa0b3 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:07.615069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c284a-2cea-48c9-9b78-1a20779679b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:07.615863 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c284a-2cea-48c9-9b78-1a20779679b8 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:07.616403 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c284a-2cea-48c9-9b78-1a20779679b8 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:07.616403 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c284a-2cea-48c9-9b78-1a20779679b8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:07.619277 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 253/513] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:07.620141 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33fec9-376e-4a81-87fd-db0f549aa0b3 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:07.625140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c284a-2cea-48c9-9b78-1a20779679b8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:07.626609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c284a-2cea-48c9-9b78-1a20779679b8 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:07.641992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-a65c284a-2cea-48c9-9b78-1a20779679b8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 20:26:07.641992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:07.641992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:07.641992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:07.641992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:07.641992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:07.641992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:07.641992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:07.641992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:07.641992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:07.641992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:07.641992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:07.641992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:07.641992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:07.641992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:07.641992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:07.642963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:07.642963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:07.642963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:07.642963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:07.642963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:07.642963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:07.642963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:07.642963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:07.642963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:07.642963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:07.642963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:07.642963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:07.642963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:07.642963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:07.642963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:07.642963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:07.642963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:07.642963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:07.643869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:07.643869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:07.643869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:07.643869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:07.643869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:07.643869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:07.643869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:07.643869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:07.643869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 20:26:07.643869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:07.645353 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 261/514] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:26:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:07.646034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c284a-2cea-48c9-9b78-1a20779679b8 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:07.724183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33fec9-376e-4a81-87fd-db0f549aa0b3 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:07.744062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c284a-2cea-48c9-9b78-1a20779679b8 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:07.755056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:07.758774 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80ca3e-d3a0-4324-940f-3ba410736a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:07.759429 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80ca3e-d3a0-4324-940f-3ba410736a37 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:07.759711 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80ca3e-d3a0-4324-940f-3ba410736a37 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:07.759987 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80ca3e-d3a0-4324-940f-3ba410736a37 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:07.760739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80ca3e-d3a0-4324-940f-3ba410736a37 None admin] RBAC: Adding query filter params (name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:07.761047 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80ca3e-d3a0-4324-940f-3ba410736a37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:07.762457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80ca3e-d3a0-4324-940f-3ba410736a37 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:07.770460 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 254/515] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:26:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:07.771251 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80ca3e-d3a0-4324-940f-3ba410736a37 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:07.775301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:07.779953 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e00720-4533-4b02-af0b-fc31e1895fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:07.780747 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e00720-4533-4b02-af0b-fc31e1895fec None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:07.781046 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e00720-4533-4b02-af0b-fc31e1895fec None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:07.781336 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e00720-4533-4b02-af0b-fc31e1895fec None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:07.788081 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e00720-4533-4b02-af0b-fc31e1895fec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:07.789562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e00720-4533-4b02-af0b-fc31e1895fec None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:07.804215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-40e00720-4533-4b02-af0b-fc31e1895fec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:07.804215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:07.804215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:07.804215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:07.804215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:07.804215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:07.804215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:07.804215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:07.804215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:07.804215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:07.804215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:07.804215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:07.804215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:07.804215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:07.804215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:07.805258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:07.805258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:07.805258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:07.805258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:07.805258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:07.805258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:07.805258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:07.805258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:07.805258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:07.805258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:07.805258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:07.805258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:07.805258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:07.805258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:07.805258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:07.805258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:07.805258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:07.806303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:07.806303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:07.806303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:07.806303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:07.806303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:07.806303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:07.806303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:07.806303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:07.806303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 262/516] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:07.807078 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e00720-4533-4b02-af0b-fc31e1895fec None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:07.876015 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80ca3e-d3a0-4324-940f-3ba410736a37 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:07.907629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e00720-4533-4b02-af0b-fc31e1895fec None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:07.910772 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540188a1-8ca7-42db-80e6-f10ccad8a622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:07.911580 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540188a1-8ca7-42db-80e6-f10ccad8a622 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:07.911864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540188a1-8ca7-42db-80e6-f10ccad8a622 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:07.912142 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540188a1-8ca7-42db-80e6-f10ccad8a622 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:07.923131 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540188a1-8ca7-42db-80e6-f10ccad8a622 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:07.924626 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540188a1-8ca7-42db-80e6-f10ccad8a622 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:07.946422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-540188a1-8ca7-42db-80e6-f10ccad8a622 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:07.946422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:07.946422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:07.946422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:07.946422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:07.946422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:07.946422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:07.946422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:07.946422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:07.946422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:07.946422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:07.946422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:07.946422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:07.946422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:07.946422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:07.947528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:07.947528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:07.947528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:07.947528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:07.947528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:07.947528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:07.947528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:07.947528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:07.947528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:07.947528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:07.947528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:07.947528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:07.947528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:07.947528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:07.947528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:07.947528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:07.947528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:07.948605 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:07.948605 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:07.948605 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:07.948605 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:07.948819 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83081e96-9533-49c3-bd54-414e19960829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:07.948819 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83081e96-9533-49c3-bd54-414e19960829 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:07.948982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 255/517] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:07.949057 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83081e96-9533-49c3-bd54-414e19960829 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:07.949057 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83081e96-9533-49c3-bd54-414e19960829 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:07.949203 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540188a1-8ca7-42db-80e6-f10ccad8a622 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:07.949283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83081e96-9533-49c3-bd54-414e19960829 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:07.950330 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83081e96-9533-49c3-bd54-414e19960829 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:07.971081 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 263/518] 10.222.0.71 () {60 vars in 1132 bytes} [Thu Jul 21 20:26:07 2022] POST /identity/v3/users => generated 247 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:26:07.972206 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83081e96-9533-49c3-bd54-414e19960829 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:08.010543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:08.033774 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:08.096848 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540188a1-8ca7-42db-80e6-f10ccad8a622 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:08.097735 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83081e96-9533-49c3-bd54-414e19960829 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:08.136631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034a489-fb94-4208-b399-fd259470dc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:08.137194 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034a489-fb94-4208-b399-fd259470dc84 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:08.137493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034a489-fb94-4208-b399-fd259470dc84 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:08.137799 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034a489-fb94-4208-b399-fd259470dc84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:08.152759 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034a489-fb94-4208-b399-fd259470dc84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:08.154558 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034a489-fb94-4208-b399-fd259470dc84 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:08.164368 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7b9f5-dfc0-4b97-9a5d-2d3212e0d073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:08.165266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7b9f5-dfc0-4b97-9a5d-2d3212e0d073 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:08.165625 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7b9f5-dfc0-4b97-9a5d-2d3212e0d073 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:08.165978 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7b9f5-dfc0-4b97-9a5d-2d3212e0d073 None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:08.166911 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7b9f5-dfc0-4b97-9a5d-2d3212e0d073 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:08.167244 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7b9f5-dfc0-4b97-9a5d-2d3212e0d073 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:08.168706 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7b9f5-dfc0-4b97-9a5d-2d3212e0d073 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:08.173053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-6034a489-fb94-4208-b399-fd259470dc84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:08.173053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:08.173053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:08.173053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:08.173053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:08.173053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:08.173053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:08.173053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:08.173053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:08.173053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:08.173053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:08.173053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:08.173053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:08.173053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:08.173053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:08.173929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:08.173929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:08.173929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:08.173929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:08.173929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:08.173929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:08.173929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:08.173929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:08.173929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:08.173929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:08.173929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:08.173929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:08.173929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:08.173929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:08.173929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:08.173929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:08.173929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:08.174787 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:08.174787 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:08.174787 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:08.174787 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:08.177680 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 264/519] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:08.178124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034a489-fb94-4208-b399-fd259470dc84 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:08.192219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 256/520] 10.222.0.71 () {56 vars in 1137 bytes} [Thu Jul 21 20:26:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:08.193131 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7b9f5-dfc0-4b97-9a5d-2d3212e0d073 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:08.270266 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:08.297757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034a489-fb94-4208-b399-fd259470dc84 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:08.315280 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7b9f5-dfc0-4b97-9a5d-2d3212e0d073 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:08.332408 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:08.339262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa291a1-34b3-48c1-af6a-af4cdcd542fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:08.340045 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa291a1-34b3-48c1-af6a-af4cdcd542fe None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:08.340396 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa291a1-34b3-48c1-af6a-af4cdcd542fe None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:08.340765 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa291a1-34b3-48c1-af6a-af4cdcd542fe None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:08.341744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa291a1-34b3-48c1-af6a-af4cdcd542fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:08.342135 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa291a1-34b3-48c1-af6a-af4cdcd542fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:08.343817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa291a1-34b3-48c1-af6a-af4cdcd542fe None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:08.364891 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 265/521] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:08.365838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa291a1-34b3-48c1-af6a-af4cdcd542fe None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:08.376622 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfd284-2030-4e34-9d3e-57936af758de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:08.377786 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfd284-2030-4e34-9d3e-57936af758de None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:08.378264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfd284-2030-4e34-9d3e-57936af758de None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:08.378718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfd284-2030-4e34-9d3e-57936af758de None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:08.379902 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfd284-2030-4e34-9d3e-57936af758de None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:08.380411 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfd284-2030-4e34-9d3e-57936af758de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:08.382280 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfd284-2030-4e34-9d3e-57936af758de None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:08.397231 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 257/522] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:08.398732 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfd284-2030-4e34-9d3e-57936af758de None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:08.506559 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa291a1-34b3-48c1-af6a-af4cdcd542fe None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:08.531632 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfd284-2030-4e34-9d3e-57936af758de None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:08.545824 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e707292-6922-4da9-96ba-da540b68c617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:08.546603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e707292-6922-4da9-96ba-da540b68c617 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:08.546888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e707292-6922-4da9-96ba-da540b68c617 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:08.547157 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e707292-6922-4da9-96ba-da540b68c617 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:08.547890 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e707292-6922-4da9-96ba-da540b68c617 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:08.548192 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e707292-6922-4da9-96ba-da540b68c617 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:08.549622 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e707292-6922-4da9-96ba-da540b68c617 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:08.557114 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:08.567293 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 266/523] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:08.567293 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e707292-6922-4da9-96ba-da540b68c617 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:08.577955 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec6230-41b1-4d1a-9339-685f4577aa17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:08.578622 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec6230-41b1-4d1a-9339-685f4577aa17 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:08.578911 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec6230-41b1-4d1a-9339-685f4577aa17 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:08.579187 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec6230-41b1-4d1a-9339-685f4577aa17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:08.626979 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:08.660334 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec6230-41b1-4d1a-9339-685f4577aa17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:08.661864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec6230-41b1-4d1a-9339-685f4577aa17 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:08.682237 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e707292-6922-4da9-96ba-da540b68c617 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:08.751464 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d8d5c-3a12-43ab-b269-05b12aafc8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:08.752381 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d8d5c-3a12-43ab-b269-05b12aafc8dd None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:08.752803 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d8d5c-3a12-43ab-b269-05b12aafc8dd None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:08.753240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d8d5c-3a12-43ab-b269-05b12aafc8dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:08.754777 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d8d5c-3a12-43ab-b269-05b12aafc8dd None admin] RBAC: Adding query filter params (role.id=9a48ab1bf32a45828e9e73599c980dac, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=6fbc5240f5214aaea4418052b94739e2) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:08.755294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d8d5c-3a12-43ab-b269-05b12aafc8dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:08.757880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d8d5c-3a12-43ab-b269-05b12aafc8dd None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:08.777119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 267/524] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:26:08 2022] GET /identity/v3/role_assignments?user.id=6fbc5240f5214aaea4418052b94739e2&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=9a48ab1bf32a45828e9e73599c980dac => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:08.777119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d8d5c-3a12-43ab-b269-05b12aafc8dd None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:08.820238 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 258/525] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:26:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:26:08.821332 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec6230-41b1-4d1a-9339-685f4577aa17 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:08.868691 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:08.929649 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d8d5c-3a12-43ab-b269-05b12aafc8dd None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:08.930886 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:08.977003 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec6230-41b1-4d1a-9339-685f4577aa17 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:08.985960 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581a34b-6bde-4a2c-879b-56e78a848a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:08.986941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581a34b-6bde-4a2c-879b-56e78a848a53 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:08.987303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581a34b-6bde-4a2c-879b-56e78a848a53 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:08.987663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581a34b-6bde-4a2c-879b-56e78a848a53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:08.997103 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581a34b-6bde-4a2c-879b-56e78a848a53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:08.999531 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581a34b-6bde-4a2c-879b-56e78a848a53 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:09.020775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-8581a34b-6bde-4a2c-879b-56e78a848a53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:09.020775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:09.020775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:09.020775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:09.020775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:09.020775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:09.020775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:09.020775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:09.020775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:09.020775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:09.020775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:09.020775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:09.020775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:09.020775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:09.020775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:09.021757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:09.021757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:09.021757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:09.021757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:09.021757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:09.021757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:09.021757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:09.021757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:09.021757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:09.021757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:09.021757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:09.021757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:09.021757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:09.021757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:09.021757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:09.021757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:09.021757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:09.022618 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:09.022618 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:09.022618 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:09.022618 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:09.022618 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:09.022618 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:09.022618 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:09.022618 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:09.022618 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 268/526] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:09.023554 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581a34b-6bde-4a2c-879b-56e78a848a53 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:09.025301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6281503-42c6-4c55-8a86-43ba9223dd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:09.026337 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6281503-42c6-4c55-8a86-43ba9223dd9f None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:09.027053 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6281503-42c6-4c55-8a86-43ba9223dd9f None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:09.027532 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6281503-42c6-4c55-8a86-43ba9223dd9f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:09.036440 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6281503-42c6-4c55-8a86-43ba9223dd9f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:09.038317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6281503-42c6-4c55-8a86-43ba9223dd9f None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:09.054793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-f6281503-42c6-4c55-8a86-43ba9223dd9f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 20:26:09.054793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:09.054793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:09.054793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:09.054793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:09.054793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:09.054793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:09.054793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:09.054793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:09.054793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:09.054793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:09.054793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:09.054793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:09.054793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:09.054793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:09.054793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:09.055837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:09.055837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:09.055837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:09.055837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:09.055837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:09.055837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:09.055837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:09.055837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:09.055837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:09.055837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:09.055837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:09.055837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:09.055837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:09.055837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:09.055837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:09.055837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:09.055837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:09.055837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:09.056736 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:09.056736 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:09.056736 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:09.056736 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:09.056736 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:09.056736 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:09.056736 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:09.056736 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:09.056736 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 20:26:09.056736 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:09.059363 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 259/527] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:09.060971 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6281503-42c6-4c55-8a86-43ba9223dd9f None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:09.147693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581a34b-6bde-4a2c-879b-56e78a848a53 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:09.175568 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:09.189752 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62c44b-3c44-4d86-ab03-8a06d152b4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:09.190836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62c44b-3c44-4d86-ab03-8a06d152b4b3 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:09.191036 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6281503-42c6-4c55-8a86-43ba9223dd9f None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:09.191508 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62c44b-3c44-4d86-ab03-8a06d152b4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:09.192002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62c44b-3c44-4d86-ab03-8a06d152b4b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:09.203528 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba62c44b-3c44-4d86-ab03-8a06d152b4b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:09.205314 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba62c44b-3c44-4d86-ab03-8a06d152b4b3 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:09.223744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-ba62c44b-3c44-4d86-ab03-8a06d152b4b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:09.223744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:09.223744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:09.223744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:09.223744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:09.223744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:09.223744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:09.223744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:09.223744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:09.223744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:09.223744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:09.223744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:09.223744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:09.223744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:09.223744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:09.224941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:09.224941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:09.224941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:09.224941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:09.224941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:09.224941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:09.224941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:09.224941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:09.224941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:09.224941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:09.224941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:09.224941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:09.224941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:09.224941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:09.224941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:09.224941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:09.224941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:09.225850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:09.225850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:09.225850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:09.225850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:09.226078 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:09.232395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 269/528] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:09.232395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62c44b-3c44-4d86-ab03-8a06d152b4b3 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:09.235481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e5494-5c55-467d-bc85-598d0339426a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:09.235919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e5494-5c55-467d-bc85-598d0339426a None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:09.236208 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e5494-5c55-467d-bc85-598d0339426a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:09.236487 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e5494-5c55-467d-bc85-598d0339426a None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:09.237319 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e5494-5c55-467d-bc85-598d0339426a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:09.237636 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e5494-5c55-467d-bc85-598d0339426a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:09.239094 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e5494-5c55-467d-bc85-598d0339426a None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:09.258128 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 260/529] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:09.258809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e5494-5c55-467d-bc85-598d0339426a None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:09.367943 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62c44b-3c44-4d86-ab03-8a06d152b4b3 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:09.375634 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e5494-5c55-467d-bc85-598d0339426a None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:09.413663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f485e8-ce3d-4808-b555-3a619e279598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:09.414995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f485e8-ce3d-4808-b555-3a619e279598 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:09.415285 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f485e8-ce3d-4808-b555-3a619e279598 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:09.415568 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f485e8-ce3d-4808-b555-3a619e279598 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:09.416448 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f485e8-ce3d-4808-b555-3a619e279598 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:09.416720 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f485e8-ce3d-4808-b555-3a619e279598 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:09.418194 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f485e8-ce3d-4808-b555-3a619e279598 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:09.419150 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3adf60-f710-4b8a-9b1d-955cc578e39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:09.420157 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3adf60-f710-4b8a-9b1d-955cc578e39e None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:09.420540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3adf60-f710-4b8a-9b1d-955cc578e39e None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:09.420916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3adf60-f710-4b8a-9b1d-955cc578e39e None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:09.421775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3adf60-f710-4b8a-9b1d-955cc578e39e None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:09.422199 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3adf60-f710-4b8a-9b1d-955cc578e39e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:09.423649 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3adf60-f710-4b8a-9b1d-955cc578e39e None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:09.438116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 261/530] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:09.439622 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3adf60-f710-4b8a-9b1d-955cc578e39e None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:09.468820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 270/531] 10.222.0.71 () {56 vars in 1139 bytes} [Thu Jul 21 20:26:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:09.469726 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f485e8-ce3d-4808-b555-3a619e279598 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:09.508292 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:09.519003 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:09.584101 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3adf60-f710-4b8a-9b1d-955cc578e39e None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:09.593651 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f485e8-ce3d-4808-b555-3a619e279598 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:09.631020 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11c3c2-2012-4b43-b816-771ea8a7928a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:09.631778 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11c3c2-2012-4b43-b816-771ea8a7928a None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:09.631993 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218b4cd-5b7a-48eb-8a5e-2f533494575f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:09.632200 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11c3c2-2012-4b43-b816-771ea8a7928a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:09.632546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11c3c2-2012-4b43-b816-771ea8a7928a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:09.632966 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218b4cd-5b7a-48eb-8a5e-2f533494575f None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:09.633353 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218b4cd-5b7a-48eb-8a5e-2f533494575f None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:09.633628 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11c3c2-2012-4b43-b816-771ea8a7928a None admin] RBAC: Adding query filter params (role.id=9a48ab1bf32a45828e9e73599c980dac, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=ee2460e6a7e34a6881984e4f7ed294c1) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:09.633736 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218b4cd-5b7a-48eb-8a5e-2f533494575f None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:09.634067 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11c3c2-2012-4b43-b816-771ea8a7928a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:09.635694 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11c3c2-2012-4b43-b816-771ea8a7928a None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:09.641905 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218b4cd-5b7a-48eb-8a5e-2f533494575f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:09.642803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 262/532] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:26:09 2022] GET /identity/v3/role_assignments?user.id=ee2460e6a7e34a6881984e4f7ed294c1&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=9a48ab1bf32a45828e9e73599c980dac => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:09.643761 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11c3c2-2012-4b43-b816-771ea8a7928a None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:09.643889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218b4cd-5b7a-48eb-8a5e-2f533494575f None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:09.674256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-d218b4cd-5b7a-48eb-8a5e-2f533494575f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:09.674256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:09.674256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:09.674256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:09.674256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:09.674256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:09.674256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:09.674256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:09.674256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:09.674256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:09.674256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:09.674256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:09.674256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:09.674256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:09.674256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:09.675348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:09.675348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:09.675348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:09.675348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:09.675348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:09.675348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:09.675348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:09.675348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:09.675348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:09.675348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:09.675348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:09.675348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:09.675348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:09.675348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:09.675348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:09.675348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:09.675348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:09.676418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:09.676418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:09.676418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:09.676418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:09.676418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:09.676418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:09.676418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:09.676418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:09.681519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 271/533] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:09.690175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218b4cd-5b7a-48eb-8a5e-2f533494575f None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:09.866822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11c3c2-2012-4b43-b816-771ea8a7928a None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:09.887486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:09.890091 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:09.907602 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218b4cd-5b7a-48eb-8a5e-2f533494575f None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:09.913370 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8fa38-67c9-4eca-9a9c-3fd5cfcecd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:09.914192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8fa38-67c9-4eca-9a9c-3fd5cfcecd42 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:09.914631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8fa38-67c9-4eca-9a9c-3fd5cfcecd42 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:09.915029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8fa38-67c9-4eca-9a9c-3fd5cfcecd42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:09.952673 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6ebee-a582-43c6-a383-d50ac4efea6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:09.953629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6ebee-a582-43c6-a383-d50ac4efea6a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:09.954057 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6ebee-a582-43c6-a383-d50ac4efea6a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:09.954501 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6ebee-a582-43c6-a383-d50ac4efea6a None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:09.955453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6ebee-a582-43c6-a383-d50ac4efea6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:09.955927 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6ebee-a582-43c6-a383-d50ac4efea6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:09.957932 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6ebee-a582-43c6-a383-d50ac4efea6a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:09.976737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 272/534] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:09.985366 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6ebee-a582-43c6-a383-d50ac4efea6a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:10.048824 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8fa38-67c9-4eca-9a9c-3fd5cfcecd42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:10.050829 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8fa38-67c9-4eca-9a9c-3fd5cfcecd42 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:10.169471 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6ebee-a582-43c6-a383-d50ac4efea6a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:10.202599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 263/535] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:26:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:26:10.204269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1ee62-d428-431f-8c1d-c9198448db09 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:10.204691 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1ee62-d428-431f-8c1d-c9198448db09 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:10.205165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1ee62-d428-431f-8c1d-c9198448db09 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:10.206886 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 264/536] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:10.213261 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1ee62-d428-431f-8c1d-c9198448db09 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:10.218462 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271dc62-4eb5-45a0-9679-0cd7d1ea45cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:10.219478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271dc62-4eb5-45a0-9679-0cd7d1ea45cf None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:10.219784 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271dc62-4eb5-45a0-9679-0cd7d1ea45cf None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:10.220240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271dc62-4eb5-45a0-9679-0cd7d1ea45cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:10.221181 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271dc62-4eb5-45a0-9679-0cd7d1ea45cf None admin] RBAC: Adding query filter params (role.id=9a48ab1bf32a45828e9e73599c980dac, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=34db429cd31549f78daf5b1cbf6f0bf0) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:10.221522 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271dc62-4eb5-45a0-9679-0cd7d1ea45cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:10.223248 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271dc62-4eb5-45a0-9679-0cd7d1ea45cf None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:10.230466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 273/537] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:26:09 2022] GET /identity/v3/role_assignments?user.id=34db429cd31549f78daf5b1cbf6f0bf0&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=9a48ab1bf32a45828e9e73599c980dac => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:10.231911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22d717-a819-42a5-99d1-62de76a9814c None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:10.232201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22d717-a819-42a5-99d1-62de76a9814c None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:10.232504 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22d717-a819-42a5-99d1-62de76a9814c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:10.251313 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:10.253937 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-4a22d717-a819-42a5-99d1-62de76a9814c None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:10.255595 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-4a22d717-a819-42a5-99d1-62de76a9814c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:10.283518 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:10.343782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 274/538] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:10.371247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22d717-a819-42a5-99d1-62de76a9814c None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:10.451473 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1ee62-d428-431f-8c1d-c9198448db09 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:10.508122 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa53c60-714b-4a5b-be90-7cae563a183d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:10.508946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa53c60-714b-4a5b-be90-7cae563a183d None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:10.509259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa53c60-714b-4a5b-be90-7cae563a183d None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:10.509537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa53c60-714b-4a5b-be90-7cae563a183d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:10.524933 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22d717-a819-42a5-99d1-62de76a9814c None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:10.531670 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa53c60-714b-4a5b-be90-7cae563a183d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:10.533185 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa53c60-714b-4a5b-be90-7cae563a183d None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:10.575852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-6fa53c60-714b-4a5b-be90-7cae563a183d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:10.575852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:10.575852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:10.575852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:10.575852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:10.575852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:10.575852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:10.575852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:10.575852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:10.575852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:10.575852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:10.575852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:10.575852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:10.575852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:10.575852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:10.576812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:10.576812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:10.576812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:10.576812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:10.576812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:10.576812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:10.576812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:10.576812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:10.576812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:10.576812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:10.576812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:10.576812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:10.576812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:10.576812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:10.576812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:10.576812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:10.576812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:10.577724 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:10.577724 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:10.577724 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:10.577724 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:10.577724 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 265/539] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:26:10.584889 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa53c60-714b-4a5b-be90-7cae563a183d None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:10.602863 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16538e-9601-4821-9c93-05c0d585e015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:10.604039 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16538e-9601-4821-9c93-05c0d585e015 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:10.604554 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16538e-9601-4821-9c93-05c0d585e015 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:10.605034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16538e-9601-4821-9c93-05c0d585e015 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:10.635776 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16538e-9601-4821-9c93-05c0d585e015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:10.653101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-9b16538e-9601-4821-9c93-05c0d585e015 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:10.653101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:10.653101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:10.653101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:10.653101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:10.653101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:10.653101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:10.653101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:10.653101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:10.653101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:10.653101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:10.653101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:10.653101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:10.653101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:10.653101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:10.653101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:10.654170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:10.654170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:10.654170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:10.654170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:10.654170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:10.654170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:10.654170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:10.654170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:10.654170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:10.654170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:10.654170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:10.654170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:10.654170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:10.654170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:10.654170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:10.654170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:10.654972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:10.654972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:10.654972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:10.654972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:10.654972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:10.654972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:10.654972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:10.654972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:10.654972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:10.654972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:10.654972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:10.654972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:10.657837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 275/540] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:10.663868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16538e-9601-4821-9c93-05c0d585e015 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:10.665435 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:10.685118 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:10.706279 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa53c60-714b-4a5b-be90-7cae563a183d None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:10.768777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140276e-b935-40f5-a572-abdc857d2891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:10.769401 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140276e-b935-40f5-a572-abdc857d2891 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:10.769684 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140276e-b935-40f5-a572-abdc857d2891 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:10.769956 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140276e-b935-40f5-a572-abdc857d2891 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:10.770696 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140276e-b935-40f5-a572-abdc857d2891 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:10.771179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140276e-b935-40f5-a572-abdc857d2891 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:10.772464 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140276e-b935-40f5-a572-abdc857d2891 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:10.806490 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 266/541] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:10.811733 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140276e-b935-40f5-a572-abdc857d2891 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:10.855200 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16538e-9601-4821-9c93-05c0d585e015 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:10.926374 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957f1ef-6bd6-4d54-8261-0b730c29c9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:10.927073 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957f1ef-6bd6-4d54-8261-0b730c29c9f5 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:10.927358 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957f1ef-6bd6-4d54-8261-0b730c29c9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:10.927636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957f1ef-6bd6-4d54-8261-0b730c29c9f5 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:10.928393 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957f1ef-6bd6-4d54-8261-0b730c29c9f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:10.928702 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957f1ef-6bd6-4d54-8261-0b730c29c9f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:10.930136 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957f1ef-6bd6-4d54-8261-0b730c29c9f5 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:10.956819 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 276/542] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:26:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:10.962316 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957f1ef-6bd6-4d54-8261-0b730c29c9f5 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:11.027218 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140276e-b935-40f5-a572-abdc857d2891 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:11.069611 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:11.074071 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0e8a4-59c7-4a6e-bc44-256d59ea8c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:11.075347 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0e8a4-59c7-4a6e-bc44-256d59ea8c15 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:11.076290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0e8a4-59c7-4a6e-bc44-256d59ea8c15 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:11.076583 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0e8a4-59c7-4a6e-bc44-256d59ea8c15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:11.083580 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:11.086625 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0e8a4-59c7-4a6e-bc44-256d59ea8c15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:11.088124 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0e8a4-59c7-4a6e-bc44-256d59ea8c15 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:11.111868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-dba0e8a4-59c7-4a6e-bc44-256d59ea8c15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:11.111868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:11.111868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:11.111868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:11.111868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:11.111868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:11.111868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:11.111868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:11.111868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:11.111868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:11.111868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:11.111868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:11.111868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:11.111868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:11.111868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:11.112788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:11.112788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:11.112788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:11.112788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:11.112788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:11.112788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:11.112788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:11.112788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:11.112788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:11.112788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:11.112788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:11.112788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:11.112788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:11.112788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:11.112788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:11.112788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:11.112788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:11.113704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:11.113704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:11.113704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:11.113704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:11.113704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:11.113704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:11.113704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:11.113704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:11.113704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 267/543] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:26:11.114862 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a0bd4-cbd4-48d8-b906-fd820a003229 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:11.115152 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a0bd4-cbd4-48d8-b906-fd820a003229 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:11.115424 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a0bd4-cbd4-48d8-b906-fd820a003229 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:11.116855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 268/544] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:11.123137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a0bd4-cbd4-48d8-b906-fd820a003229 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:11.133244 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957f1ef-6bd6-4d54-8261-0b730c29c9f5 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:11.172756 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cef363-5fba-474f-b288-b156e363f727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:11.173583 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cef363-5fba-474f-b288-b156e363f727 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:11.173861 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cef363-5fba-474f-b288-b156e363f727 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:11.174134 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cef363-5fba-474f-b288-b156e363f727 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:11.186944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cef363-5fba-474f-b288-b156e363f727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:11.188428 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cef363-5fba-474f-b288-b156e363f727 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:11.209219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-55cef363-5fba-474f-b288-b156e363f727 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:11.209219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:11.209219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:11.209219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:11.209219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:11.209219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:11.209219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:11.209219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:11.209219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:11.209219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:11.209219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:11.209219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:11.209219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:11.209219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:11.209219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:11.210902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:11.210902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:11.210902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:11.210902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:11.210902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:11.210902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:11.210902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:11.210902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:11.210902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:11.210902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:11.210902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:11.210902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:11.210902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:11.210902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:11.210902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:11.210902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:11.210902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:11.212663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:11.212663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:11.212663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:11.212663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:11.212663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 277/545] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:11.212663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e53c99-b221-4758-9f4c-39995b1c8859 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:11.212663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e53c99-b221-4758-9f4c-39995b1c8859 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:11.212663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e53c99-b221-4758-9f4c-39995b1c8859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:11.232325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-f1e53c99-b221-4758-9f4c-39995b1c8859 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:11.233567 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f1e53c99-b221-4758-9f4c-39995b1c8859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:11.290169 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a0bd4-cbd4-48d8-b906-fd820a003229 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:11.330913 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ad87c-3a23-40eb-b9dd-838a4fb2214f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:11.331574 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ad87c-3a23-40eb-b9dd-838a4fb2214f None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:11.331858 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ad87c-3a23-40eb-b9dd-838a4fb2214f None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:11.332131 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ad87c-3a23-40eb-b9dd-838a4fb2214f None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:11.332929 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ad87c-3a23-40eb-b9dd-838a4fb2214f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:11.333264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ad87c-3a23-40eb-b9dd-838a4fb2214f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:11.334780 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ad87c-3a23-40eb-b9dd-838a4fb2214f None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:11.353204 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 269/546] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:11.354161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ad87c-3a23-40eb-b9dd-838a4fb2214f None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:11.374593 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 278/547] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:11.377135 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e53c99-b221-4758-9f4c-39995b1c8859 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:11.439063 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:11.492570 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:11.522057 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ad87c-3a23-40eb-b9dd-838a4fb2214f None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:11.548986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e53c99-b221-4758-9f4c-39995b1c8859 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:11.566633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937af04-b36f-4a8a-b974-bcb55e97e1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:11.567357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937af04-b36f-4a8a-b974-bcb55e97e1dd None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:11.567710 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937af04-b36f-4a8a-b974-bcb55e97e1dd None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:11.567984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937af04-b36f-4a8a-b974-bcb55e97e1dd None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:11.568880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937af04-b36f-4a8a-b974-bcb55e97e1dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:11.569247 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937af04-b36f-4a8a-b974-bcb55e97e1dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:11.570878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937af04-b36f-4a8a-b974-bcb55e97e1dd None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:11.586925 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 270/548] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:11.587986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937af04-b36f-4a8a-b974-bcb55e97e1dd None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:11.593701 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9acd4-f868-4c70-beeb-e93cb0807c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:11.595067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9acd4-f868-4c70-beeb-e93cb0807c5f None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:11.595690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9acd4-f868-4c70-beeb-e93cb0807c5f None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:11.596236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9acd4-f868-4c70-beeb-e93cb0807c5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:11.598303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9acd4-f868-4c70-beeb-e93cb0807c5f None admin] RBAC: Adding query filter params (role.id=9a48ab1bf32a45828e9e73599c980dac, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=03eee8ef92314296b7d5c14c52f0d080) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:11.599234 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9acd4-f868-4c70-beeb-e93cb0807c5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:11.602998 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9acd4-f868-4c70-beeb-e93cb0807c5f None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:11.616327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 279/549] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:26:11 2022] GET /identity/v3/role_assignments?user.id=03eee8ef92314296b7d5c14c52f0d080&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=9a48ab1bf32a45828e9e73599c980dac => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:11.618467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9acd4-f868-4c70-beeb-e93cb0807c5f None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:11.724244 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937af04-b36f-4a8a-b974-bcb55e97e1dd None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:11.736807 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:11.768129 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305b739-52a2-4634-ab7a-67d30e139bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:11.768838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305b739-52a2-4634-ab7a-67d30e139bed None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:11.769309 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305b739-52a2-4634-ab7a-67d30e139bed None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:11.769574 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305b739-52a2-4634-ab7a-67d30e139bed None admin] PATH_INFO: `/v3/roles/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:11.783464 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305b739-52a2-4634-ab7a-67d30e139bed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:11.791805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-8305b739-52a2-4634-ab7a-67d30e139bed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:11.791805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:11.791805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:11.791805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:11.791805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:11.791805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:11.791805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:11.791805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:11.791805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:11.791805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:11.791805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:11.791805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:11.791805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:11.791805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:11.791805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:11.791805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:11.794472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:11.794472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:11.794472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:11.794472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:11.794472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:11.794472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:11.794472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:11.794472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:11.794472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:11.794472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:11.794472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:11.794472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:11.794472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:11.794472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:11.794472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:11.794472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:11.796103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:11.796103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:11.796103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:11.796103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:11.796103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:11.796103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:11.796103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:11.796103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:11.796103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:11.796103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:11.796103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:11.796103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:11.797712 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 271/550] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:11.800245 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305b739-52a2-4634-ab7a-67d30e139bed None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:11.908635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9acd4-f868-4c70-beeb-e93cb0807c5f None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:11.924881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:11.959288 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305b739-52a2-4634-ab7a-67d30e139bed None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:11.978185 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ee0ff-7e58-4322-9ad0-198c2b033632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:11.979275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ee0ff-7e58-4322-9ad0-198c2b033632 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:11.979792 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ee0ff-7e58-4322-9ad0-198c2b033632 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:11.980073 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ee0ff-7e58-4322-9ad0-198c2b033632 None admin] PATH_INFO: `/v3/users/swift` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:12.001315 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ccf01-b03f-4d5d-8109-320fdbfd7727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:12.002468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ee0ff-7e58-4322-9ad0-198c2b033632 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:12.002695 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ccf01-b03f-4d5d-8109-320fdbfd7727 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:12.003180 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ccf01-b03f-4d5d-8109-320fdbfd7727 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:12.003689 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ccf01-b03f-4d5d-8109-320fdbfd7727 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:12.003945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ee0ff-7e58-4322-9ad0-198c2b033632 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:12.004976 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ccf01-b03f-4d5d-8109-320fdbfd7727 None admin] RBAC: Adding query filter params (name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:12.005518 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ccf01-b03f-4d5d-8109-320fdbfd7727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:12.007219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ccf01-b03f-4d5d-8109-320fdbfd7727 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:12.017987 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 272/551] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:26:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:12.034820 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ccf01-b03f-4d5d-8109-320fdbfd7727 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:12.035326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-7d7ee0ff-7e58-4322-9ad0-198c2b033632 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:26:12.035326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:12.035326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:12.035326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:12.035326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:12.035326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:12.035326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:12.035326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:12.035326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:12.035326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:12.035326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:12.035326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:12.035326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:12.035326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:12.035326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:12.035326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:12.036335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:12.036335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:12.036335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:12.036335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:12.036335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:12.036335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:12.036335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:12.036335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:12.036335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:12.036335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:12.036335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:12.036335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:12.036335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:12.036335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:12.036335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:12.036335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:12.036335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:12.036335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:12.037395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:12.037395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:12.037395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:12.037395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:12.037395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:12.037395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:12.037395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:12.037395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:12.037395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:26:12.037395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:12.037395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 280/552] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:26:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:12.040995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a993f8-5737-4b46-b8e8-2081eee80f11 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:12.041435 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a993f8-5737-4b46-b8e8-2081eee80f11 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:12.041931 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a993f8-5737-4b46-b8e8-2081eee80f11 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:12.043325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 281/553] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:12.044123 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a993f8-5737-4b46-b8e8-2081eee80f11 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:12.084432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:12.166674 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ccf01-b03f-4d5d-8109-320fdbfd7727 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:12.178579 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a993f8-5737-4b46-b8e8-2081eee80f11 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:12.218031 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ee326-b97b-4e01-b845-2ee20329fb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:12.218246 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d01ed-397b-4e8d-bade-d0f289ab0a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:12.218818 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ee326-b97b-4e01-b845-2ee20329fb9c None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:12.219172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ee326-b97b-4e01-b845-2ee20329fb9c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:12.219519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ee326-b97b-4e01-b845-2ee20329fb9c None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:12.219861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d01ed-397b-4e8d-bade-d0f289ab0a9b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:12.220319 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d01ed-397b-4e8d-bade-d0f289ab0a9b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:12.220574 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ee326-b97b-4e01-b845-2ee20329fb9c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:12.220949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ee326-b97b-4e01-b845-2ee20329fb9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:12.221160 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d01ed-397b-4e8d-bade-d0f289ab0a9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:12.222846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ee326-b97b-4e01-b845-2ee20329fb9c None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:12.237539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d01ed-397b-4e8d-bade-d0f289ab0a9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:12.239357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d01ed-397b-4e8d-bade-d0f289ab0a9b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:12.246597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 282/554] 10.222.0.71 () {56 vars in 1135 bytes} [Thu Jul 21 20:26:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:12.248694 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b45e5-81cd-4c7b-b992-d9135afb9874 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:12.249055 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b45e5-81cd-4c7b-b992-d9135afb9874 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:12.249446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b45e5-81cd-4c7b-b992-d9135afb9874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:12.264867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-ea8d01ed-397b-4e8d-bade-d0f289ab0a9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:12.264867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:12.264867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:12.264867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:12.264867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:12.264867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:12.264867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:12.264867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:12.264867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:12.264867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:12.264867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:12.264867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:12.264867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:12.264867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:12.264867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:12.265841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:12.265841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:12.265841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:12.265841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:12.265841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:12.265841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:12.265841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:12.265841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:12.265841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:12.265841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:12.265841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:12.265841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:12.265841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:12.265841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:12.265841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:12.265841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:12.265841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:12.266727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:12.266727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:12.266727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:12.266727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:12.267592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-222b45e5-81cd-4c7b-b992-d9135afb9874 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:12.269003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-222b45e5-81cd-4c7b-b992-d9135afb9874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:12.269464 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 273/555] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:26:12.270475 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d01ed-397b-4e8d-bade-d0f289ab0a9b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:12.300497 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:12.358155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 283/556] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:12.365753 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b45e5-81cd-4c7b-b992-d9135afb9874 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:12.394257 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:12.426351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d01ed-397b-4e8d-bade-d0f289ab0a9b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:12.491217 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35490e45-6f28-4b2e-8666-926889cd074a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:12.492573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35490e45-6f28-4b2e-8666-926889cd074a None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:12.493381 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35490e45-6f28-4b2e-8666-926889cd074a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:12.494055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35490e45-6f28-4b2e-8666-926889cd074a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:12.520647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b45e5-81cd-4c7b-b992-d9135afb9874 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:12.565217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55f7e0-2bc3-4f46-a306-23e0c787bb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:12.566219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55f7e0-2bc3-4f46-a306-23e0c787bb4a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:12.566656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55f7e0-2bc3-4f46-a306-23e0c787bb4a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:12.567129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55f7e0-2bc3-4f46-a306-23e0c787bb4a None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:12.568903 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55f7e0-2bc3-4f46-a306-23e0c787bb4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:12.569891 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55f7e0-2bc3-4f46-a306-23e0c787bb4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:12.572269 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55f7e0-2bc3-4f46-a306-23e0c787bb4a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:12.597856 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 284/557] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:12.597856 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55f7e0-2bc3-4f46-a306-23e0c787bb4a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:12.622127 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35490e45-6f28-4b2e-8666-926889cd074a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:12.625488 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:12.628530 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35490e45-6f28-4b2e-8666-926889cd074a None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:12.719354 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55f7e0-2bc3-4f46-a306-23e0c787bb4a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:12.739140 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:12.761570 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563316ce-35ed-41c2-ac65-08631f5b04c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:12.762422 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563316ce-35ed-41c2-ac65-08631f5b04c5 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:12.762745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563316ce-35ed-41c2-ac65-08631f5b04c5 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:12.763576 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563316ce-35ed-41c2-ac65-08631f5b04c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:12.765277 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 274/558] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:26:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:26:12.770335 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35490e45-6f28-4b2e-8666-926889cd074a None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:12.776188 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563316ce-35ed-41c2-ac65-08631f5b04c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:12.784222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-563316ce-35ed-41c2-ac65-08631f5b04c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:12.784222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:12.784222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:12.784222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:12.784222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:12.784222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:12.784222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:12.784222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:12.784222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:12.784222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:12.784222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:12.784222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:12.784222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:12.784222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:12.784222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:12.784222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:12.787140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:12.787140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:12.787140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:12.787140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:12.787140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:12.787140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:12.787140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:12.787140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:12.787140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:12.787140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:12.787140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:12.787140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:12.787140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:12.787140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:12.787140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:12.787140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:12.788587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:12.788587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:12.788587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:12.788587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:12.788587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:12.788587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:12.788587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:12.788587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:12.788587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:12.788587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:12.788587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:12.788587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:12.788587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 285/559] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:12.788587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e716f3-a24b-44f6-af3b-1009d461368c None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:12.790541 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e716f3-a24b-44f6-af3b-1009d461368c None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:12.790541 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e716f3-a24b-44f6-af3b-1009d461368c None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:12.790541 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 286/560] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:12.790541 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e716f3-a24b-44f6-af3b-1009d461368c None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:12.900880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35490e45-6f28-4b2e-8666-926889cd074a None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:12.922358 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:12.941978 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e1edd-778a-48a5-a5e5-0989f90caba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:12.942903 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e1edd-778a-48a5-a5e5-0989f90caba6 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:12.943194 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e1edd-778a-48a5-a5e5-0989f90caba6 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:12.943508 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e1edd-778a-48a5-a5e5-0989f90caba6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:12.945332 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e716f3-a24b-44f6-af3b-1009d461368c None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:12.952981 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e1edd-778a-48a5-a5e5-0989f90caba6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:12.954524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e1edd-778a-48a5-a5e5-0989f90caba6 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:12.970478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-926e1edd-778a-48a5-a5e5-0989f90caba6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:26:12.970478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:12.970478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:12.970478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:12.970478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:12.970478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:12.970478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:12.970478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:12.970478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:12.970478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:12.970478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:12.970478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:12.970478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:12.970478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:12.970478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:12.970478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:12.971828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:12.971828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:12.971828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:12.971828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:12.971828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:12.971828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:12.971828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:12.971828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:12.971828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:12.971828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:12.971828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:12.971828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:12.971828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:12.971828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:12.971828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:12.971828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:12.971828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:12.971828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:12.973138 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:12.973138 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:12.973138 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:12.973138 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:12.973138 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:12.973138 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:12.973138 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:12.973138 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:12.973138 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:26:12.973138 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:12.976772 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 275/561] 10.222.0.71 () {56 vars in 1093 bytes} [Thu Jul 21 20:26:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:12.977845 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e1edd-778a-48a5-a5e5-0989f90caba6 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:12.994766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989b50b-d4c4-4dfc-b102-894749d057a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:12.995995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989b50b-d4c4-4dfc-b102-894749d057a7 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:12.996419 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989b50b-d4c4-4dfc-b102-894749d057a7 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:12.996831 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989b50b-d4c4-4dfc-b102-894749d057a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:13.011434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989b50b-d4c4-4dfc-b102-894749d057a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:13.012955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989b50b-d4c4-4dfc-b102-894749d057a7 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:13.031349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-7989b50b-d4c4-4dfc-b102-894749d057a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:13.031349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:13.031349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:13.031349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:13.031349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:13.031349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:13.031349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:13.031349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:13.031349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:13.031349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:13.031349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:13.031349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:13.031349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:13.031349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:13.031349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:13.032553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:13.032553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:13.032553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:13.032553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:13.032553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:13.032553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:13.032553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:13.032553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:13.032553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:13.032553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:13.032553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:13.032553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:13.032553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:13.032553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:13.032553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:13.032553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:13.032553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:13.033820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:13.033820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:13.033820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:13.033820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:13.034104 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:13.034182 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 287/562] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:13.034182 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863b191-8015-4e37-a96a-4dcb12c61494 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:13.035934 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863b191-8015-4e37-a96a-4dcb12c61494 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:13.036237 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863b191-8015-4e37-a96a-4dcb12c61494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:13.055136 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-5863b191-8015-4e37-a96a-4dcb12c61494 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:13.056463 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-5863b191-8015-4e37-a96a-4dcb12c61494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:13.117096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e1edd-778a-48a5-a5e5-0989f90caba6 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:13.149116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 288/563] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:13.149116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863b191-8015-4e37-a96a-4dcb12c61494 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:13.169837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55308235-817a-4aeb-acf0-eabb79ce6008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:13.171028 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55308235-817a-4aeb-acf0-eabb79ce6008 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:13.171553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55308235-817a-4aeb-acf0-eabb79ce6008 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:13.172060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55308235-817a-4aeb-acf0-eabb79ce6008 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:13.173739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55308235-817a-4aeb-acf0-eabb79ce6008 None admin] RBAC: Adding query filter params (name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:13.174302 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55308235-817a-4aeb-acf0-eabb79ce6008 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:13.176821 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55308235-817a-4aeb-acf0-eabb79ce6008 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:13.193107 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 276/564] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:26:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:13.193107 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55308235-817a-4aeb-acf0-eabb79ce6008 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:13.232647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:13.279376 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863b191-8015-4e37-a96a-4dcb12c61494 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:13.333674 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d4779-77f0-4784-9fd9-ba95d9babfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:13.334566 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d4779-77f0-4784-9fd9-ba95d9babfd9 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:13.334935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d4779-77f0-4784-9fd9-ba95d9babfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:13.335321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d4779-77f0-4784-9fd9-ba95d9babfd9 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:13.336371 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d4779-77f0-4784-9fd9-ba95d9babfd9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:13.336690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d4779-77f0-4784-9fd9-ba95d9babfd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:13.338484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d4779-77f0-4784-9fd9-ba95d9babfd9 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:13.339511 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55308235-817a-4aeb-acf0-eabb79ce6008 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:13.344404 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:13.372811 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 289/565] 10.222.0.71 () {56 vars in 1133 bytes} [Thu Jul 21 20:26:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:13.372811 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d4779-77f0-4784-9fd9-ba95d9babfd9 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:13.398257 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7b4bb-de5b-40d5-8ac7-e926ea05fb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:13.399330 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7b4bb-de5b-40d5-8ac7-e926ea05fb60 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:13.399767 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7b4bb-de5b-40d5-8ac7-e926ea05fb60 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:13.400188 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7b4bb-de5b-40d5-8ac7-e926ea05fb60 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:13.401467 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad7b4bb-de5b-40d5-8ac7-e926ea05fb60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:13.401937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad7b4bb-de5b-40d5-8ac7-e926ea05fb60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:13.404312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad7b4bb-de5b-40d5-8ac7-e926ea05fb60 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:13.430957 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 277/566] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:13.430957 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7b4bb-de5b-40d5-8ac7-e926ea05fb60 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:13.543617 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d4779-77f0-4784-9fd9-ba95d9babfd9 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:13.570796 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7b4bb-de5b-40d5-8ac7-e926ea05fb60 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:13.588773 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:13.608615 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf3598-fdb0-4c03-b6fc-6bb77a64a9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:13.610023 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf3598-fdb0-4c03-b6fc-6bb77a64a9cd None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:13.610398 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf3598-fdb0-4c03-b6fc-6bb77a64a9cd None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:13.610681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf3598-fdb0-4c03-b6fc-6bb77a64a9cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:13.618916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b4ce1-aa2e-4c86-8597-325c71e75385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:13.619705 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b4ce1-aa2e-4c86-8597-325c71e75385 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:13.619987 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b4ce1-aa2e-4c86-8597-325c71e75385 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:13.620401 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b4ce1-aa2e-4c86-8597-325c71e75385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:13.623615 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf3598-fdb0-4c03-b6fc-6bb77a64a9cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:13.631743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-5baf3598-fdb0-4c03-b6fc-6bb77a64a9cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:13.631743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:13.631743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:13.631743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:13.631743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:13.631743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:13.631743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:13.631743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:13.631743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:13.631743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:13.631743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:13.631743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:13.631743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:13.631743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:13.631743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:13.631743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:13.633853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:13.633853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:13.633853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:13.633853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:13.633853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:13.633853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:13.633853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:13.633853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:13.633853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:13.633853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:13.633853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:13.633853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:13.633853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:13.633853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:13.633853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:13.633853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:13.636018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:13.636018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:13.636018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:13.636018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:13.636018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:13.636018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:13.636018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:13.636018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:13.636018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:13.636018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:13.636018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:13.636018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:13.637339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b4ce1-aa2e-4c86-8597-325c71e75385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:13.637339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b4ce1-aa2e-4c86-8597-325c71e75385 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:13.639643 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 290/567] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:13.639643 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf3598-fdb0-4c03-b6fc-6bb77a64a9cd None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:13.658850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-8e0b4ce1-aa2e-4c86-8597-325c71e75385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:13.658850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:13.658850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:13.658850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:13.658850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:13.658850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:13.658850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:13.658850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:13.658850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:13.658850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:13.658850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:13.658850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:13.658850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:13.658850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:13.658850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:13.660134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:13.660134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:13.660134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:13.660134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:13.660134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:13.660134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:13.660134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:13.660134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:13.660134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:13.660134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:13.660134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:13.660134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:13.660134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:13.660134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:13.660134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:13.660134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:13.660134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:13.661475 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:13.661475 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:13.661475 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:13.661475 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:13.664959 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 278/568] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:13.665867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b4ce1-aa2e-4c86-8597-325c71e75385 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:13.686982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:13.755161 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf3598-fdb0-4c03-b6fc-6bb77a64a9cd None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:13.788007 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b4ce1-aa2e-4c86-8597-325c71e75385 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:13.792940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4cfff-7aee-4e8e-a29b-b68ba83b5847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:13.794500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4cfff-7aee-4e8e-a29b-b68ba83b5847 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:13.794971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4cfff-7aee-4e8e-a29b-b68ba83b5847 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:13.795256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4cfff-7aee-4e8e-a29b-b68ba83b5847 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:13.806992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4cfff-7aee-4e8e-a29b-b68ba83b5847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:13.808487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4cfff-7aee-4e8e-a29b-b68ba83b5847 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:13.831186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-18d4cfff-7aee-4e8e-a29b-b68ba83b5847 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:13.831186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:13.831186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:13.831186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:13.831186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:13.831186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:13.831186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:13.831186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:13.831186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:13.831186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:13.831186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:13.831186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:13.831186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:13.831186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:13.831186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:13.832144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:13.832144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:13.832144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:13.832144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:13.832144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:13.832144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:13.832144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:13.832144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:13.832144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:13.832144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:13.832144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:13.832144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:13.832144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:13.832144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:13.832144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:13.832144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:13.832144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:13.837884 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:13.837884 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:13.837884 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:13.837884 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:13.838154 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc224ff-975b-4b46-b0f3-5ff6bbb5a7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:13.838154 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc224ff-975b-4b46-b0f3-5ff6bbb5a7e0 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:13.838154 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc224ff-975b-4b46-b0f3-5ff6bbb5a7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:13.838401 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 291/569] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:13.838483 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc224ff-975b-4b46-b0f3-5ff6bbb5a7e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:13.838564 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4cfff-7aee-4e8e-a29b-b68ba83b5847 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:13.845215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc224ff-975b-4b46-b0f3-5ff6bbb5a7e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:13.846800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc224ff-975b-4b46-b0f3-5ff6bbb5a7e0 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:13.862861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-cfc224ff-975b-4b46-b0f3-5ff6bbb5a7e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:13.862861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:13.862861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:13.862861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:13.862861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:13.862861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:13.862861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:13.862861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:13.862861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:13.862861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:13.862861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:13.862861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:13.862861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:13.862861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:13.862861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:13.864070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:13.864070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:13.864070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:13.864070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:13.864070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:13.864070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:13.864070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:13.864070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:13.864070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:13.864070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:13.864070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:13.864070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:13.864070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:13.864070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:13.864070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:13.864070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:13.864070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:13.871832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:13.871832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:13.871832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:13.871832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:13.871832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:13.871832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:13.871832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:13.871832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:13.871832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 279/570] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:13.871832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc224ff-975b-4b46-b0f3-5ff6bbb5a7e0 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:13.895141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:13.990185 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:14.000252 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4cfff-7aee-4e8e-a29b-b68ba83b5847 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:14.026372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc224ff-975b-4b46-b0f3-5ff6bbb5a7e0 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:14.042468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e69421-5015-45d0-a5eb-13e42f148ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:14.043160 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e69421-5015-45d0-a5eb-13e42f148ed0 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:14.043439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e69421-5015-45d0-a5eb-13e42f148ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:14.043713 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e69421-5015-45d0-a5eb-13e42f148ed0 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:14.044533 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e69421-5015-45d0-a5eb-13e42f148ed0 None admin] RBAC: Adding query filter params (name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:14.044844 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e69421-5015-45d0-a5eb-13e42f148ed0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:14.046282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e69421-5015-45d0-a5eb-13e42f148ed0 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:14.055364 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 292/571] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:26:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:14.056655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e69421-5015-45d0-a5eb-13e42f148ed0 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:14.065478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8cc2f-910d-442e-b6c0-3e590a23cc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:14.066161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8cc2f-910d-442e-b6c0-3e590a23cc28 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:14.066444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8cc2f-910d-442e-b6c0-3e590a23cc28 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:14.066724 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8cc2f-910d-442e-b6c0-3e590a23cc28 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:14.067467 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8cc2f-910d-442e-b6c0-3e590a23cc28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:14.067779 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8cc2f-910d-442e-b6c0-3e590a23cc28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:14.069243 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8cc2f-910d-442e-b6c0-3e590a23cc28 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:14.087630 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 280/572] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:14.088893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8cc2f-910d-442e-b6c0-3e590a23cc28 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:14.172212 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:14.178881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e69421-5015-45d0-a5eb-13e42f148ed0 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:14.205713 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8cc2f-910d-442e-b6c0-3e590a23cc28 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:14.223890 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ebc43-63fa-49c2-b994-0df9d5524c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:14.224565 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ebc43-63fa-49c2-b994-0df9d5524c48 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:14.225133 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ebc43-63fa-49c2-b994-0df9d5524c48 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:14.225413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ebc43-63fa-49c2-b994-0df9d5524c48 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:14.226159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ebc43-63fa-49c2-b994-0df9d5524c48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:14.226471 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ebc43-63fa-49c2-b994-0df9d5524c48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:14.227913 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ebc43-63fa-49c2-b994-0df9d5524c48 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:14.241555 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 293/573] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:14.243258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3aca8-cef3-4c42-9247-36754a0e1f4f None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:14.243258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3aca8-cef3-4c42-9247-36754a0e1f4f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:14.243521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3aca8-cef3-4c42-9247-36754a0e1f4f None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:14.244898 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 294/574] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:14.245766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3aca8-cef3-4c42-9247-36754a0e1f4f None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:14.258413 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18013adb-ddf7-4e3d-9cfa-af5b4eafc431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:14.259095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18013adb-ddf7-4e3d-9cfa-af5b4eafc431 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:14.259450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18013adb-ddf7-4e3d-9cfa-af5b4eafc431 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:14.259746 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18013adb-ddf7-4e3d-9cfa-af5b4eafc431 None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:14.260571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18013adb-ddf7-4e3d-9cfa-af5b4eafc431 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:14.260888 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18013adb-ddf7-4e3d-9cfa-af5b4eafc431 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:14.262405 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18013adb-ddf7-4e3d-9cfa-af5b4eafc431 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:14.265391 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:14.275704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 281/575] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:14.276821 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18013adb-ddf7-4e3d-9cfa-af5b4eafc431 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:14.373391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3aca8-cef3-4c42-9247-36754a0e1f4f None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:14.396364 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18013adb-ddf7-4e3d-9cfa-af5b4eafc431 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:14.410236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02eaf2-6c1f-4f46-ab8d-3264c0f412d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:14.411038 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02eaf2-6c1f-4f46-ab8d-3264c0f412d2 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:14.411301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02eaf2-6c1f-4f46-ab8d-3264c0f412d2 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:14.411574 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02eaf2-6c1f-4f46-ab8d-3264c0f412d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:14.423425 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a02eaf2-6c1f-4f46-ab8d-3264c0f412d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:14.424925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a02eaf2-6c1f-4f46-ab8d-3264c0f412d2 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:14.431957 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3896352-7607-4913-9afb-0d4caea6379a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:14.432870 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3896352-7607-4913-9afb-0d4caea6379a None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:14.433191 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3896352-7607-4913-9afb-0d4caea6379a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:14.433475 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3896352-7607-4913-9afb-0d4caea6379a None admin] PATH_INFO: `/v3/users/glance` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:14.442128 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3896352-7607-4913-9afb-0d4caea6379a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:14.443412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-4a02eaf2-6c1f-4f46-ab8d-3264c0f412d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:14.443412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:14.443412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:14.443412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:14.443412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:14.443412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:14.443412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:14.443412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:14.443412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:14.443412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:14.443412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:14.443412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:14.443412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:14.443412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:14.443412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:14.444467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:14.444467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:14.444467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:14.444467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:14.444467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:14.444467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:14.444467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:14.444467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:14.444467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:14.444467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:14.444467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:14.444467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:14.444467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:14.444467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:14.444467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:14.444467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:14.444467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:14.445430 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:14.445430 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:14.445430 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:14.445430 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:14.445669 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3896352-7607-4913-9afb-0d4caea6379a None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:14.445741 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 295/576] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:14.445741 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02eaf2-6c1f-4f46-ab8d-3264c0f412d2 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:14.457496 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:14.461553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-e3896352-7607-4913-9afb-0d4caea6379a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 20:26:14.461553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:14.461553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:14.461553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:14.461553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:14.461553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:14.461553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:14.461553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:14.461553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:14.461553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:14.461553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:14.461553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:14.461553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:14.461553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:14.461553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:14.461553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:14.462540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:14.462540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:14.462540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:14.462540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:14.462540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:14.462540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:14.462540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:14.462540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:14.462540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:14.462540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:14.462540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:14.462540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:14.462540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:14.462540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:14.462540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:14.462540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:14.462540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:14.462540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:14.463510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:14.463510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:14.463510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:14.463510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:14.463510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:14.463510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:14.463510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:14.463510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:14.463510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 20:26:14.463510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:14.463510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 282/577] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:26:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:14.464257 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae92682-8e76-4701-a4bb-b2d3eea97802 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:14.464418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae92682-8e76-4701-a4bb-b2d3eea97802 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:14.464691 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae92682-8e76-4701-a4bb-b2d3eea97802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:14.485924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-5ae92682-8e76-4701-a4bb-b2d3eea97802 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:14.487148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-5ae92682-8e76-4701-a4bb-b2d3eea97802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:14.541308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:14.555375 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02eaf2-6c1f-4f46-ab8d-3264c0f412d2 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:14.560899 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 283/578] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:14.561982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae92682-8e76-4701-a4bb-b2d3eea97802 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:14.593765 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad3e6d-86c3-4c5a-a1b7-55e5582ed995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:14.594844 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad3e6d-86c3-4c5a-a1b7-55e5582ed995 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:14.595114 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad3e6d-86c3-4c5a-a1b7-55e5582ed995 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:14.595387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad3e6d-86c3-4c5a-a1b7-55e5582ed995 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:14.603395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad3e6d-86c3-4c5a-a1b7-55e5582ed995 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:14.604875 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad3e6d-86c3-4c5a-a1b7-55e5582ed995 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:14.619493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-3dad3e6d-86c3-4c5a-a1b7-55e5582ed995 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:14.619493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:14.619493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:14.619493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:14.619493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:14.619493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:14.619493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:14.619493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:14.619493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:14.619493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:14.619493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:14.619493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:14.619493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:14.619493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:14.619493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:14.620477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:14.620477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:14.620477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:14.620477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:14.620477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:14.620477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:14.620477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:14.620477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:14.620477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:14.620477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:14.620477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:14.620477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:14.620477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:14.620477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:14.620477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:14.620477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:14.620477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:14.621442 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:14.621442 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:14.621442 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:14.621442 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:14.621442 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:14.621442 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:14.621442 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:14.621442 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:14.621442 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 296/579] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:14.621948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad3e6d-86c3-4c5a-a1b7-55e5582ed995 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:14.670248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae92682-8e76-4701-a4bb-b2d3eea97802 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:14.694283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:14.707261 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af239a41-9445-4cc3-bc99-004612a16e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:14.707923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af239a41-9445-4cc3-bc99-004612a16e6e None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:14.708207 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af239a41-9445-4cc3-bc99-004612a16e6e None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:14.708481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af239a41-9445-4cc3-bc99-004612a16e6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:14.709349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af239a41-9445-4cc3-bc99-004612a16e6e None admin] RBAC: Adding query filter params (role.id=9a48ab1bf32a45828e9e73599c980dac, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=3babb078203647e2b3ef3b2787b84e3b) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:14.709658 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af239a41-9445-4cc3-bc99-004612a16e6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:14.711118 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af239a41-9445-4cc3-bc99-004612a16e6e None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:14.717248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 284/580] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:26:14 2022] GET /identity/v3/role_assignments?user.id=3babb078203647e2b3ef3b2787b84e3b&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=9a48ab1bf32a45828e9e73599c980dac => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:14.718282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af239a41-9445-4cc3-bc99-004612a16e6e None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:14.751528 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad3e6d-86c3-4c5a-a1b7-55e5582ed995 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:14.790274 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:14.796845 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41284e69-e66b-497c-901d-c1544c5206fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:14.797809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41284e69-e66b-497c-901d-c1544c5206fa None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:14.798124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41284e69-e66b-497c-901d-c1544c5206fa None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:14.798372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41284e69-e66b-497c-901d-c1544c5206fa None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:14.799123 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41284e69-e66b-497c-901d-c1544c5206fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:14.799456 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41284e69-e66b-497c-901d-c1544c5206fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:14.801027 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41284e69-e66b-497c-901d-c1544c5206fa None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:14.814442 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 297/581] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:14.815321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41284e69-e66b-497c-901d-c1544c5206fa None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:14.864443 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af239a41-9445-4cc3-bc99-004612a16e6e None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:14.928821 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be159b1-a6ca-4f45-aff8-2da318c781f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:14.929647 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be159b1-a6ca-4f45-aff8-2da318c781f3 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:14.929929 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be159b1-a6ca-4f45-aff8-2da318c781f3 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:14.930194 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be159b1-a6ca-4f45-aff8-2da318c781f3 None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:14.931050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be159b1-a6ca-4f45-aff8-2da318c781f3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:14.931353 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be159b1-a6ca-4f45-aff8-2da318c781f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:14.932833 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be159b1-a6ca-4f45-aff8-2da318c781f3 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:14.957803 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41284e69-e66b-497c-901d-c1544c5206fa None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:14.961030 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 285/582] 10.222.0.71 () {56 vars in 1137 bytes} [Thu Jul 21 20:26:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:14.966884 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be159b1-a6ca-4f45-aff8-2da318c781f3 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:14.999744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:15.009297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240ae4c-eb4a-4cb1-9717-73fde2b23468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:15.009967 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240ae4c-eb4a-4cb1-9717-73fde2b23468 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:15.010251 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240ae4c-eb4a-4cb1-9717-73fde2b23468 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:15.010525 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240ae4c-eb4a-4cb1-9717-73fde2b23468 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:15.022079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240ae4c-eb4a-4cb1-9717-73fde2b23468 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:15.030443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-1240ae4c-eb4a-4cb1-9717-73fde2b23468 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:15.030443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:15.030443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:15.030443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:15.030443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:15.030443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:15.030443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:15.030443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:15.030443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:15.030443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:15.030443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:15.030443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:15.030443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:15.030443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:15.030443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:15.030443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:15.031942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:15.031942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:15.031942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:15.031942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:15.031942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:15.031942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:15.031942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:15.031942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:15.031942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:15.031942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:15.031942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:15.031942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:15.031942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:15.031942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:15.031942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:15.031942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:15.032850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:15.032850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:15.032850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:15.032850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:15.032850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:15.032850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:15.032850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:15.032850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:15.032850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:15.032850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:15.032850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:15.032850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:15.032850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 298/583] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:15.032850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240ae4c-eb4a-4cb1-9717-73fde2b23468 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:15.112674 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be159b1-a6ca-4f45-aff8-2da318c781f3 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:15.124774 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:15.164554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e0e37-f8ee-4a7e-bc0f-a1449e511e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:15.165273 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e0e37-f8ee-4a7e-bc0f-a1449e511e4f None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:15.165552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e0e37-f8ee-4a7e-bc0f-a1449e511e4f None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:15.165817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e0e37-f8ee-4a7e-bc0f-a1449e511e4f None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:15.166626 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e0e37-f8ee-4a7e-bc0f-a1449e511e4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:15.166930 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e0e37-f8ee-4a7e-bc0f-a1449e511e4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:15.168478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e0e37-f8ee-4a7e-bc0f-a1449e511e4f None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:15.183924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240ae4c-eb4a-4cb1-9717-73fde2b23468 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:15.184168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 286/584] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:15.184168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e0e37-f8ee-4a7e-bc0f-a1449e511e4f None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:15.234092 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e605b75-753c-41d1-b0d6-85690afd834b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:15.235457 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e605b75-753c-41d1-b0d6-85690afd834b None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:15.235773 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e605b75-753c-41d1-b0d6-85690afd834b None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:15.236105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e605b75-753c-41d1-b0d6-85690afd834b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:15.245014 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e605b75-753c-41d1-b0d6-85690afd834b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:15.246542 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e605b75-753c-41d1-b0d6-85690afd834b None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:15.266215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-3e605b75-753c-41d1-b0d6-85690afd834b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 20:26:15.266215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:15.266215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:15.266215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:15.266215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:15.266215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:15.266215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:15.266215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:15.266215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:15.266215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:15.266215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:15.266215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:15.266215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:15.266215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:15.266215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:15.266215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:15.267283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:15.267283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:15.267283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:15.267283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:15.267283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:15.267283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:15.267283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:15.267283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:15.267283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:15.267283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:15.267283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:15.267283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:15.267283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:15.267283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:15.267283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:15.267283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:15.267283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:15.267283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:15.268284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:15.268284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:15.268284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:15.268284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:15.268284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:15.268284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:15.268284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:15.268284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:15.268284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 20:26:15.268284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:15.268284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 299/585] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:26:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:15.268284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e605b75-753c-41d1-b0d6-85690afd834b None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:15.327593 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e0e37-f8ee-4a7e-bc0f-a1449e511e4f None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:15.338466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:15.369655 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ac529-92d3-4a1b-8c34-a984a95b223e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:15.370807 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac529-92d3-4a1b-8c34-a984a95b223e None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:15.371245 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac529-92d3-4a1b-8c34-a984a95b223e None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:15.371669 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac529-92d3-4a1b-8c34-a984a95b223e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:15.384484 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ac529-92d3-4a1b-8c34-a984a95b223e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:15.386088 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ac529-92d3-4a1b-8c34-a984a95b223e None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:15.404700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-348ac529-92d3-4a1b-8c34-a984a95b223e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:15.404700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:15.404700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:15.404700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:15.404700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:15.404700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:15.404700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:15.404700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:15.404700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:15.404700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:15.404700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:15.404700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:15.404700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:15.404700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:15.404700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:15.405709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:15.405709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:15.405709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:15.405709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:15.405709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:15.405709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:15.405709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:15.405709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:15.405709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:15.405709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:15.405709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:15.405709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:15.405709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:15.405709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:15.405709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:15.405709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:15.405709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:15.406625 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:15.406625 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:15.406625 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:15.406625 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:15.406625 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 287/586] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:15.406934 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ac529-92d3-4a1b-8c34-a984a95b223e None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:15.409659 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e605b75-753c-41d1-b0d6-85690afd834b None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:15.438142 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:15.452180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908abc33-0ca3-4bac-b5d0-f9608e1acb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:15.452948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908abc33-0ca3-4bac-b5d0-f9608e1acb3e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:15.453283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908abc33-0ca3-4bac-b5d0-f9608e1acb3e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:15.453717 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908abc33-0ca3-4bac-b5d0-f9608e1acb3e None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:15.454700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908abc33-0ca3-4bac-b5d0-f9608e1acb3e None admin] RBAC: Adding query filter params (name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:15.455107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908abc33-0ca3-4bac-b5d0-f9608e1acb3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:15.457062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908abc33-0ca3-4bac-b5d0-f9608e1acb3e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:15.467870 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 300/587] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:26:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:15.468721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908abc33-0ca3-4bac-b5d0-f9608e1acb3e None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:15.547409 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ac529-92d3-4a1b-8c34-a984a95b223e None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:15.584810 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c42ba-f7ef-4ab2-b21c-f0367cc6e768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:15.586024 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c42ba-f7ef-4ab2-b21c-f0367cc6e768 None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:15.586464 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c42ba-f7ef-4ab2-b21c-f0367cc6e768 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:15.586586 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908abc33-0ca3-4bac-b5d0-f9608e1acb3e None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:15.587334 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c42ba-f7ef-4ab2-b21c-f0367cc6e768 None admin] PATH_INFO: `/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/6fbc5240f5214aaea4418052b94739e2/roles/9a48ab1bf32a45828e9e73599c980dac` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:15.598296 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:15.608298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c42ba-f7ef-4ab2-b21c-f0367cc6e768 None admin] RBAC: Authorizing `identity:create_grant(project_id=37e4afa595894a9994a8e6981ebb7c41, user_id=6fbc5240f5214aaea4418052b94739e2, role_id=9a48ab1bf32a45828e9e73599c980dac)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:15.609820 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c42ba-f7ef-4ab2-b21c-f0367cc6e768 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:15.629195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 288/588] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:26:15 2022] PUT /identity/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/6fbc5240f5214aaea4418052b94739e2/roles/9a48ab1bf32a45828e9e73599c980dac => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:26:15.629195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c42ba-f7ef-4ab2-b21c-f0367cc6e768 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:15.640121 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a7623-841e-4b9e-9fad-480941618ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:15.641926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a7623-841e-4b9e-9fad-480941618ecc None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:15.642810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a7623-841e-4b9e-9fad-480941618ecc None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:15.643356 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a7623-841e-4b9e-9fad-480941618ecc None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:15.644831 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a7623-841e-4b9e-9fad-480941618ecc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:15.646077 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a7623-841e-4b9e-9fad-480941618ecc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:15.648599 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a7623-841e-4b9e-9fad-480941618ecc None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:15.677527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 301/589] 10.222.0.71 () {56 vars in 1137 bytes} [Thu Jul 21 20:26:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:15.678903 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a7623-841e-4b9e-9fad-480941618ecc None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:15.749525 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:15.814603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a7623-841e-4b9e-9fad-480941618ecc None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:15.814927 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c42ba-f7ef-4ab2-b21c-f0367cc6e768 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:15.861306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175312c-81cd-44ff-b927-7c2967e10cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:15.862258 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175312c-81cd-44ff-b927-7c2967e10cfe None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:15.862640 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175312c-81cd-44ff-b927-7c2967e10cfe None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:15.862934 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175312c-81cd-44ff-b927-7c2967e10cfe None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:15.863844 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175312c-81cd-44ff-b927-7c2967e10cfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:15.864159 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175312c-81cd-44ff-b927-7c2967e10cfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:15.865780 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175312c-81cd-44ff-b927-7c2967e10cfe None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:15.870847 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfad91-98bc-4768-bf0d-70b106b259c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:15.872783 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfad91-98bc-4768-bf0d-70b106b259c0 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:15.873811 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfad91-98bc-4768-bf0d-70b106b259c0 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:15.874612 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfad91-98bc-4768-bf0d-70b106b259c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:15.887278 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 289/590] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:15.888672 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175312c-81cd-44ff-b927-7c2967e10cfe None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:15.896995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfad91-98bc-4768-bf0d-70b106b259c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:15.900403 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfad91-98bc-4768-bf0d-70b106b259c0 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:15.935417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-44cfad91-98bc-4768-bf0d-70b106b259c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:15.935417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:15.935417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:15.935417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:15.935417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:15.935417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:15.935417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:15.935417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:15.935417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:15.935417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:15.935417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:15.935417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:15.935417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:15.935417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:15.935417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:15.936979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:15.936979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:15.936979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:15.936979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:15.936979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:15.936979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:15.936979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:15.936979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:15.936979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:15.936979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:15.936979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:15.936979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:15.936979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:15.936979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:15.936979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:15.936979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:15.936979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:15.938862 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:15.938862 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:15.938862 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:15.938862 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:15.941998 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 302/591] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:15.948357 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfad91-98bc-4768-bf0d-70b106b259c0 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:15.985281 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:16.038869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175312c-81cd-44ff-b927-7c2967e10cfe None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:16.074363 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:16.082415 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56a72c-1585-4620-844d-a748c74bf1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:16.083394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56a72c-1585-4620-844d-a748c74bf1b0 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:16.083781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56a72c-1585-4620-844d-a748c74bf1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:16.084160 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56a72c-1585-4620-844d-a748c74bf1b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:16.097520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56a72c-1585-4620-844d-a748c74bf1b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:16.099183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56a72c-1585-4620-844d-a748c74bf1b0 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:16.108585 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfad91-98bc-4768-bf0d-70b106b259c0 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:16.120809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-9b56a72c-1585-4620-844d-a748c74bf1b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:16.120809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:16.120809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:16.120809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:16.120809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:16.120809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:16.120809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:16.120809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:16.120809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:16.120809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:16.120809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:16.120809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:16.120809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:16.120809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:16.120809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:16.122006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:16.122006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:16.122006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:16.122006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:16.122006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:16.122006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:16.122006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:16.122006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:16.122006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:16.122006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:16.122006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:16.122006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:16.122006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:16.122006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:16.122006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:16.122006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:16.122006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:16.123224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:16.123224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:16.123224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:16.123224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:16.124826 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 290/592] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:16.129964 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56a72c-1585-4620-844d-a748c74bf1b0 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:16.161597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d042b-8f1d-4d92-99fb-a79cdad793a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:16.162958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d042b-8f1d-4d92-99fb-a79cdad793a6 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:16.163408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d042b-8f1d-4d92-99fb-a79cdad793a6 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:16.164097 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d042b-8f1d-4d92-99fb-a79cdad793a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:16.175042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d042b-8f1d-4d92-99fb-a79cdad793a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:16.176799 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d042b-8f1d-4d92-99fb-a79cdad793a6 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:16.196241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-2b0d042b-8f1d-4d92-99fb-a79cdad793a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:16.196241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:16.196241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:16.196241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:16.196241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:16.196241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:16.196241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:16.196241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:16.196241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:16.196241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:16.196241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:16.196241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:16.196241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:16.196241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:16.196241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:16.197409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:16.197409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:16.197409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:16.197409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:16.197409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:16.197409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:16.197409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:16.197409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:16.197409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:16.197409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:16.197409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:16.197409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:16.197409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:16.197409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:16.197409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:16.197409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:16.197409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:16.198484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:16.198484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:16.198484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:16.198484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:16.198484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:16.198484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:16.198484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:16.198484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:16.205578 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 303/593] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:16.205578 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d042b-8f1d-4d92-99fb-a79cdad793a6 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:16.263727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56a72c-1585-4620-844d-a748c74bf1b0 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:16.310847 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc4a16-eb90-4e17-8ff2-20fa866ea9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:16.312203 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc4a16-eb90-4e17-8ff2-20fa866ea9d6 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:16.312640 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc4a16-eb90-4e17-8ff2-20fa866ea9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:16.313019 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc4a16-eb90-4e17-8ff2-20fa866ea9d6 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:16.314124 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc4a16-eb90-4e17-8ff2-20fa866ea9d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:16.314556 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc4a16-eb90-4e17-8ff2-20fa866ea9d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:16.316576 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc4a16-eb90-4e17-8ff2-20fa866ea9d6 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:16.324816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:16.334650 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 291/594] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:16.340083 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc4a16-eb90-4e17-8ff2-20fa866ea9d6 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:16.353556 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d042b-8f1d-4d92-99fb-a79cdad793a6 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:16.406862 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:16.419397 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a340a3-109a-49ec-b5dc-c02febd55ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:16.420583 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a340a3-109a-49ec-b5dc-c02febd55ace None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:16.421018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a340a3-109a-49ec-b5dc-c02febd55ace None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:16.421675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a340a3-109a-49ec-b5dc-c02febd55ace None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:16.423231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a340a3-109a-49ec-b5dc-c02febd55ace None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:16.423603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a340a3-109a-49ec-b5dc-c02febd55ace None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:16.425711 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a340a3-109a-49ec-b5dc-c02febd55ace None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:16.456597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 304/595] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:16.459882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a340a3-109a-49ec-b5dc-c02febd55ace None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:16.483645 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc4a16-eb90-4e17-8ff2-20fa866ea9d6 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:16.529096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dddd87-e12a-46d8-bd61-130d8860035a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:16.530018 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dddd87-e12a-46d8-bd61-130d8860035a None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:16.530626 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dddd87-e12a-46d8-bd61-130d8860035a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:16.531062 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dddd87-e12a-46d8-bd61-130d8860035a None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:16.532357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dddd87-e12a-46d8-bd61-130d8860035a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:16.533081 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dddd87-e12a-46d8-bd61-130d8860035a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:16.535102 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dddd87-e12a-46d8-bd61-130d8860035a None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:16.555404 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 292/596] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:16.555404 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dddd87-e12a-46d8-bd61-130d8860035a None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:16.593792 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a340a3-109a-49ec-b5dc-c02febd55ace None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:16.643539 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8928aa5-2fc1-48ba-877a-03f22ad1384e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:16.644927 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8928aa5-2fc1-48ba-877a-03f22ad1384e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:16.645794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8928aa5-2fc1-48ba-877a-03f22ad1384e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:16.646495 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8928aa5-2fc1-48ba-877a-03f22ad1384e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:16.663117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8928aa5-2fc1-48ba-877a-03f22ad1384e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:16.666450 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8928aa5-2fc1-48ba-877a-03f22ad1384e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:16.674655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:16.689609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-b8928aa5-2fc1-48ba-877a-03f22ad1384e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 20:26:16.689609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:16.689609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:16.689609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:16.689609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:16.689609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:16.689609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:16.689609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:16.689609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:16.689609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:16.689609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:16.689609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:16.689609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:16.689609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:16.689609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:16.689609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:16.690656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:16.690656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:16.690656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:16.690656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:16.690656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:16.690656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:16.690656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:16.690656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:16.690656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:16.690656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:16.690656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:16.690656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:16.690656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:16.690656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:16.690656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:16.690656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:16.690656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:16.690656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:16.691601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:16.691601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:16.691601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:16.691601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:16.691601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:16.691601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:16.691601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:16.691601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:16.691601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 20:26:16.691601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:16.695596 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 305/597] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:16.696495 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8928aa5-2fc1-48ba-877a-03f22ad1384e None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:16.715769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dddd87-e12a-46d8-bd61-130d8860035a None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:16.717670 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:16.771274 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2eb18d-f9fb-4d76-9bb7-04a767ce7417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:16.772181 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2eb18d-f9fb-4d76-9bb7-04a767ce7417 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:16.772505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2eb18d-f9fb-4d76-9bb7-04a767ce7417 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:16.772838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2eb18d-f9fb-4d76-9bb7-04a767ce7417 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:16.785016 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2eb18d-f9fb-4d76-9bb7-04a767ce7417 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:16.786632 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2eb18d-f9fb-4d76-9bb7-04a767ce7417 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:16.803249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-5b2eb18d-f9fb-4d76-9bb7-04a767ce7417 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:16.803249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:16.803249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:16.803249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:16.803249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:16.803249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:16.803249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:16.803249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:16.803249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:16.803249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:16.803249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:16.803249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:16.803249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:16.803249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:16.803249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:16.804317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:16.804317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:16.804317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:16.804317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:16.804317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:16.804317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:16.804317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:16.804317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:16.804317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:16.804317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:16.804317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:16.804317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:16.804317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:16.804317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:16.804317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:16.804317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:16.804317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:16.805272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:16.805272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:16.805272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:16.805272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:16.805272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:16.805272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:16.805272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:16.805272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:16.809366 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 293/598] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:16.810432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2eb18d-f9fb-4d76-9bb7-04a767ce7417 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:16.854491 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8928aa5-2fc1-48ba-877a-03f22ad1384e None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:16.895969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4313f1-d516-4822-91ed-19da4bd01d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:16.896969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4313f1-d516-4822-91ed-19da4bd01d45 None admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:16.897277 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4313f1-d516-4822-91ed-19da4bd01d45 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:16.897549 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4313f1-d516-4822-91ed-19da4bd01d45 None admin] PATH_INFO: `/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/ee2460e6a7e34a6881984e4f7ed294c1/roles/9a48ab1bf32a45828e9e73599c980dac` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:16.922806 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4313f1-d516-4822-91ed-19da4bd01d45 None admin] RBAC: Authorizing `identity:create_grant(project_id=37e4afa595894a9994a8e6981ebb7c41, user_id=ee2460e6a7e34a6881984e4f7ed294c1, role_id=9a48ab1bf32a45828e9e73599c980dac)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:16.925043 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4313f1-d516-4822-91ed-19da4bd01d45 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:16.932287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2eb18d-f9fb-4d76-9bb7-04a767ce7417 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:16.948973 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 306/599] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:26:16 2022] PUT /identity/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/ee2460e6a7e34a6881984e4f7ed294c1/roles/9a48ab1bf32a45828e9e73599c980dac => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:26:16.954114 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4313f1-d516-4822-91ed-19da4bd01d45 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:16.977096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e026b-f33d-490b-ac99-3f03b70ade88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:16.977939 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e026b-f33d-490b-ac99-3f03b70ade88 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:16.978128 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e026b-f33d-490b-ac99-3f03b70ade88 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:16.978397 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e026b-f33d-490b-ac99-3f03b70ade88 None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:16.979289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e026b-f33d-490b-ac99-3f03b70ade88 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:16.979602 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e026b-f33d-490b-ac99-3f03b70ade88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:16.982226 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e026b-f33d-490b-ac99-3f03b70ade88 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:17.010751 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 294/600] 10.222.0.71 () {56 vars in 1139 bytes} [Thu Jul 21 20:26:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:17.012004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eefc97-2f60-4be4-8aad-fd4a03f672b9 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:17.012308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eefc97-2f60-4be4-8aad-fd4a03f672b9 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:17.012608 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eefc97-2f60-4be4-8aad-fd4a03f672b9 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:17.021249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 295/601] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:17.021249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eefc97-2f60-4be4-8aad-fd4a03f672b9 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:17.043755 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:17.055282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:17.144828 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4313f1-d516-4822-91ed-19da4bd01d45 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:17.178382 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eefc97-2f60-4be4-8aad-fd4a03f672b9 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:17.206056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6f51d-5bba-4512-ad5a-7f9db5a14319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:17.207256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6f51d-5bba-4512-ad5a-7f9db5a14319 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:17.207964 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6f51d-5bba-4512-ad5a-7f9db5a14319 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:17.208662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6f51d-5bba-4512-ad5a-7f9db5a14319 None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:17.210211 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6f51d-5bba-4512-ad5a-7f9db5a14319 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:17.210855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6f51d-5bba-4512-ad5a-7f9db5a14319 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:17.213321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6f51d-5bba-4512-ad5a-7f9db5a14319 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:17.229422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac148a25-a910-4ce5-bf7f-03f6c68003b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:17.230537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac148a25-a910-4ce5-bf7f-03f6c68003b8 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:17.230978 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac148a25-a910-4ce5-bf7f-03f6c68003b8 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:17.231461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac148a25-a910-4ce5-bf7f-03f6c68003b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:17.232282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 307/602] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:17.237802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc9587-fdc1-44b4-bdbe-9090984a8a7f None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:17.237802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc9587-fdc1-44b4-bdbe-9090984a8a7f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:17.237802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc9587-fdc1-44b4-bdbe-9090984a8a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:17.249720 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac148a25-a910-4ce5-bf7f-03f6c68003b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:17.251607 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac148a25-a910-4ce5-bf7f-03f6c68003b8 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:17.255583 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-bbbc9587-fdc1-44b4-bdbe-9090984a8a7f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:17.256903 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-bbbc9587-fdc1-44b4-bdbe-9090984a8a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:17.272698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-ac148a25-a910-4ce5-bf7f-03f6c68003b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:17.272698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:17.272698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:17.272698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:17.272698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:17.272698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:17.272698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:17.272698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:17.272698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:17.272698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:17.272698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:17.272698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:17.272698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:17.272698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:17.272698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:17.273773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:17.273773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:17.273773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:17.273773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:17.273773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:17.273773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:17.273773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:17.273773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:17.273773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:17.273773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:17.273773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:17.273773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:17.273773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:17.273773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:17.273773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:17.273773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:17.273773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:17.274694 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:17.274694 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:17.274694 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:17.274694 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:17.276809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 296/603] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:17.282303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac148a25-a910-4ce5-bf7f-03f6c68003b8 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:17.341548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 308/604] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:17.349345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc9587-fdc1-44b4-bdbe-9090984a8a7f None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:17.408290 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:17.417458 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:17.472375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac148a25-a910-4ce5-bf7f-03f6c68003b8 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:17.519116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc9587-fdc1-44b4-bdbe-9090984a8a7f None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:17.519887 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172c567-c148-4af2-9084-646c36ab7ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:17.521109 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172c567-c148-4af2-9084-646c36ab7ee7 None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:17.521713 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172c567-c148-4af2-9084-646c36ab7ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:17.522204 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172c567-c148-4af2-9084-646c36ab7ee7 None admin] PATH_INFO: `/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/34db429cd31549f78daf5b1cbf6f0bf0/roles/9a48ab1bf32a45828e9e73599c980dac` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:17.546938 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172c567-c148-4af2-9084-646c36ab7ee7 None admin] RBAC: Authorizing `identity:create_grant(project_id=37e4afa595894a9994a8e6981ebb7c41, user_id=34db429cd31549f78daf5b1cbf6f0bf0, role_id=9a48ab1bf32a45828e9e73599c980dac)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:17.548845 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172c567-c148-4af2-9084-646c36ab7ee7 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:17.576638 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 297/605] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:26:17 2022] PUT /identity/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/34db429cd31549f78daf5b1cbf6f0bf0/roles/9a48ab1bf32a45828e9e73599c980dac => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:26:17.576638 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172c567-c148-4af2-9084-646c36ab7ee7 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:17.612066 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc41326-418e-44b1-8b1b-366d41a2feef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:17.612835 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc41326-418e-44b1-8b1b-366d41a2feef None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:17.613016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc41326-418e-44b1-8b1b-366d41a2feef None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:17.613315 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc41326-418e-44b1-8b1b-366d41a2feef None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:17.614057 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc41326-418e-44b1-8b1b-366d41a2feef None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:17.614412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc41326-418e-44b1-8b1b-366d41a2feef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:17.615864 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc41326-418e-44b1-8b1b-366d41a2feef None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:17.640708 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 309/606] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:17.648700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc41326-418e-44b1-8b1b-366d41a2feef None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:17.815341 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:17.819391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc41326-418e-44b1-8b1b-366d41a2feef None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:17.865205 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23d21c-854c-4bdf-9c9f-89b51433c9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:17.866426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23d21c-854c-4bdf-9c9f-89b51433c9d3 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:17.866708 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23d21c-854c-4bdf-9c9f-89b51433c9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:17.866982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23d21c-854c-4bdf-9c9f-89b51433c9d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:17.876514 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d23d21c-854c-4bdf-9c9f-89b51433c9d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:17.879476 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d23d21c-854c-4bdf-9c9f-89b51433c9d3 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:17.884016 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172c567-c148-4af2-9084-646c36ab7ee7 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:17.909444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-0d23d21c-854c-4bdf-9c9f-89b51433c9d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:17.909444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:17.909444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:17.909444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:17.909444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:17.909444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:17.909444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:17.909444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:17.909444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:17.909444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:17.909444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:17.909444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:17.909444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:17.909444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:17.909444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:17.910868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:17.910868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:17.910868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:17.910868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:17.910868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:17.910868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:17.910868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:17.910868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:17.910868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:17.910868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:17.910868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:17.910868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:17.910868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:17.910868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:17.910868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:17.910868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:17.910868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:17.912236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:17.912236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:17.912236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:17.912236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:17.912236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:17.912236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:17.912236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:17.912236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:17.915248 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 310/607] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:17.916697 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626e5d5-66fb-4921-bc57-b36a79abace9 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:17.916978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626e5d5-66fb-4921-bc57-b36a79abace9 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:17.917302 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626e5d5-66fb-4921-bc57-b36a79abace9 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:17.920515 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 311/608] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:17.926467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626e5d5-66fb-4921-bc57-b36a79abace9 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:17.939966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:17.957339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24f191-9b7b-4e8e-b814-b97f9b4dc121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:17.958644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24f191-9b7b-4e8e-b814-b97f9b4dc121 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:17.959566 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24f191-9b7b-4e8e-b814-b97f9b4dc121 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:17.960326 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24f191-9b7b-4e8e-b814-b97f9b4dc121 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:17.978033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a24f191-9b7b-4e8e-b814-b97f9b4dc121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:17.988439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-0a24f191-9b7b-4e8e-b814-b97f9b4dc121 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:17.988439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:17.988439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:17.988439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:17.988439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:17.988439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:17.988439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:17.988439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:17.988439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:17.988439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:17.988439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:17.988439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:17.988439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:17.988439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:17.988439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:17.988439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:17.989500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:17.989500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:17.989500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:17.989500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:17.989500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:17.989500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:17.989500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:17.989500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:17.989500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:17.989500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:17.989500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:17.989500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:17.989500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:17.989500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:17.989500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:17.989500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:17.990470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:17.990470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:17.990470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:17.990470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:17.990470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:17.990470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:17.990470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:17.990470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:17.990470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:17.990470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:17.990470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:17.990470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:17.992068 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 298/609] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:17.993230 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910aa25-53bd-4ee1-86ba-8f0e5001a2a0 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:17.993570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910aa25-53bd-4ee1-86ba-8f0e5001a2a0 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:17.993793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910aa25-53bd-4ee1-86ba-8f0e5001a2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:18.012267 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-2910aa25-53bd-4ee1-86ba-8f0e5001a2a0 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:18.013697 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-2910aa25-53bd-4ee1-86ba-8f0e5001a2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:18.081812 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626e5d5-66fb-4921-bc57-b36a79abace9 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:18.104569 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 299/610] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:18.105841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910aa25-53bd-4ee1-86ba-8f0e5001a2a0 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:18.130048 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a35693-5ffb-446e-a49f-0215c50c002f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:18.130721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a35693-5ffb-446e-a49f-0215c50c002f None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:18.131005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a35693-5ffb-446e-a49f-0215c50c002f None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:18.131282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a35693-5ffb-446e-a49f-0215c50c002f None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:18.132710 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a35693-5ffb-446e-a49f-0215c50c002f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:18.133028 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a35693-5ffb-446e-a49f-0215c50c002f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:18.135297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a35693-5ffb-446e-a49f-0215c50c002f None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:18.153938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 312/611] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:18.154949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a35693-5ffb-446e-a49f-0215c50c002f None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:18.187653 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:18.244203 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910aa25-53bd-4ee1-86ba-8f0e5001a2a0 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:18.266514 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:18.293207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a35693-5ffb-446e-a49f-0215c50c002f None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:18.295477 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8857426-f4c8-48f9-9217-72d861ac3449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:18.296269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8857426-f4c8-48f9-9217-72d861ac3449 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:18.296657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8857426-f4c8-48f9-9217-72d861ac3449 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:18.297037 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8857426-f4c8-48f9-9217-72d861ac3449 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:18.298232 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8857426-f4c8-48f9-9217-72d861ac3449 None admin] RBAC: Adding query filter params (name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:18.298232 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8857426-f4c8-48f9-9217-72d861ac3449 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:18.299882 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8857426-f4c8-48f9-9217-72d861ac3449 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:18.310494 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 300/612] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:26:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:18.311554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8857426-f4c8-48f9-9217-72d861ac3449 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:18.342309 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8ecad-10a5-46fe-a909-d801f91367a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:18.343639 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8ecad-10a5-46fe-a909-d801f91367a4 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:18.344521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8ecad-10a5-46fe-a909-d801f91367a4 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:18.345093 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8ecad-10a5-46fe-a909-d801f91367a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:18.366337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a8ecad-10a5-46fe-a909-d801f91367a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:18.377978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-39a8ecad-10a5-46fe-a909-d801f91367a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:18.377978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:18.377978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:18.377978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:18.377978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:18.377978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:18.377978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:18.377978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:18.377978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:18.377978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:18.377978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:18.377978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:18.377978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:18.377978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:18.377978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:18.377978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:18.379180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:18.379180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:18.379180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:18.379180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:18.379180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:18.379180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:18.379180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:18.379180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:18.379180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:18.379180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:18.379180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:18.379180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:18.379180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:18.379180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:18.379180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:18.379180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:18.380238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:18.380238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:18.380238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:18.380238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:18.380238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:18.380238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:18.380238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:18.380238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:18.380238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:18.380238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:18.380238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:18.380238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:18.382825 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 313/613] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:18.384271 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8ecad-10a5-46fe-a909-d801f91367a4 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:18.443585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8857426-f4c8-48f9-9217-72d861ac3449 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:18.493782 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916bd4f-2bbf-4bc1-a3bd-0e8cbc1e84b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:18.495156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:18.496203 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916bd4f-2bbf-4bc1-a3bd-0e8cbc1e84b9 None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:18.496869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916bd4f-2bbf-4bc1-a3bd-0e8cbc1e84b9 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:18.497974 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916bd4f-2bbf-4bc1-a3bd-0e8cbc1e84b9 None admin] PATH_INFO: `/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/03eee8ef92314296b7d5c14c52f0d080/roles/9a48ab1bf32a45828e9e73599c980dac` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:18.517860 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916bd4f-2bbf-4bc1-a3bd-0e8cbc1e84b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=37e4afa595894a9994a8e6981ebb7c41, user_id=03eee8ef92314296b7d5c14c52f0d080, role_id=9a48ab1bf32a45828e9e73599c980dac)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:18.519929 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916bd4f-2bbf-4bc1-a3bd-0e8cbc1e84b9 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:18.534387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8ecad-10a5-46fe-a909-d801f91367a4 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:18.540710 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 301/614] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:26:18 2022] PUT /identity/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/03eee8ef92314296b7d5c14c52f0d080/roles/9a48ab1bf32a45828e9e73599c980dac => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:26:18.542018 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916bd4f-2bbf-4bc1-a3bd-0e8cbc1e84b9 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:18.595547 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf427f1-b718-420b-a9df-5ce00ab0293e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:18.596627 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf427f1-b718-420b-a9df-5ce00ab0293e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:18.597166 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf427f1-b718-420b-a9df-5ce00ab0293e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:18.597557 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf427f1-b718-420b-a9df-5ce00ab0293e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:18.605479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:18.624313 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf427f1-b718-420b-a9df-5ce00ab0293e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:18.626705 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf427f1-b718-420b-a9df-5ce00ab0293e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:18.649255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-0cf427f1-b718-420b-a9df-5ce00ab0293e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:18.649255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:18.649255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:18.649255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:18.649255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:18.649255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:18.649255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:18.649255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:18.649255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:18.649255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:18.649255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:18.649255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:18.649255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:18.649255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:18.649255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:18.650268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:18.650268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:18.650268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:18.650268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:18.650268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:18.650268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:18.650268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:18.650268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:18.650268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:18.650268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:18.650268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:18.650268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:18.650268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:18.650268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:18.650268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:18.650268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:18.650268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:18.651176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:18.651176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:18.651176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:18.651176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:18.654020 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 314/615] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:18.662240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf427f1-b718-420b-a9df-5ce00ab0293e None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:18.792057 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916bd4f-2bbf-4bc1-a3bd-0e8cbc1e84b9 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:18.806137 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf427f1-b718-420b-a9df-5ce00ab0293e None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:18.856278 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:18.863178 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e056f-388d-4add-b81d-8cf3440a4993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:18.863857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e056f-388d-4add-b81d-8cf3440a4993 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:18.864138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e056f-388d-4add-b81d-8cf3440a4993 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:18.864411 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e056f-388d-4add-b81d-8cf3440a4993 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:18.865186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e056f-388d-4add-b81d-8cf3440a4993 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:18.865497 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e056f-388d-4add-b81d-8cf3440a4993 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:18.867791 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e056f-388d-4add-b81d-8cf3440a4993 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:18.879271 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68cf71-1a5a-49f8-9d27-1ffeaf3b9ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:18.880279 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68cf71-1a5a-49f8-9d27-1ffeaf3b9ba2 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:18.880563 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68cf71-1a5a-49f8-9d27-1ffeaf3b9ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:18.880846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68cf71-1a5a-49f8-9d27-1ffeaf3b9ba2 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:18.881655 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68cf71-1a5a-49f8-9d27-1ffeaf3b9ba2 None admin] RBAC: Adding query filter params (name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:18.881898 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 315/616] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:18.882019 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68cf71-1a5a-49f8-9d27-1ffeaf3b9ba2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:18.883363 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68cf71-1a5a-49f8-9d27-1ffeaf3b9ba2 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:18.887612 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e056f-388d-4add-b81d-8cf3440a4993 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:18.905084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 302/617] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:26:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:18.910670 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68cf71-1a5a-49f8-9d27-1ffeaf3b9ba2 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:19.077463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:19.105013 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68cf71-1a5a-49f8-9d27-1ffeaf3b9ba2 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:19.128552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e056f-388d-4add-b81d-8cf3440a4993 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:19.187190 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b67c6-85ce-43ef-9d9e-516bde214fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:19.188337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b67c6-85ce-43ef-9d9e-516bde214fec None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:19.188743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b67c6-85ce-43ef-9d9e-516bde214fec None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:19.189253 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b67c6-85ce-43ef-9d9e-516bde214fec None admin] PATH_INFO: `/v3/users/swift` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:19.191498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b021e09-7ed9-4767-957b-b79eeedcf0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:19.192376 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b021e09-7ed9-4767-957b-b79eeedcf0a4 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:19.192661 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b021e09-7ed9-4767-957b-b79eeedcf0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:19.192933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b021e09-7ed9-4767-957b-b79eeedcf0a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:19.199309 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b67c6-85ce-43ef-9d9e-516bde214fec None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:19.201730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b67c6-85ce-43ef-9d9e-516bde214fec None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:19.219310 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b021e09-7ed9-4767-957b-b79eeedcf0a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:19.219633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-2a9b67c6-85ce-43ef-9d9e-516bde214fec None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:26:19.219633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:19.219633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:19.219633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:19.219633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:19.219633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:19.219633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:19.219633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:19.219633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:19.219633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:19.219633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:19.219633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:19.219633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:19.219633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:19.219633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:19.219633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:19.220563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:19.222225 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:19.222225 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:19.222225 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:19.222225 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:19.222225 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:19.222225 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:19.222225 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:19.222225 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:19.222225 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:26:19.222225 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:19.223047 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b021e09-7ed9-4767-957b-b79eeedcf0a4 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:19.225175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 316/618] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:26:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:19.227179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b67c6-85ce-43ef-9d9e-516bde214fec None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:19.269539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-5b021e09-7ed9-4767-957b-b79eeedcf0a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:19.269539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:19.269539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:19.269539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:19.269539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:19.269539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:19.269539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:19.269539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:19.269539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:19.269539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:19.269539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:19.269539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:19.269539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:19.269539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:19.269539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:19.270555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:19.270555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:19.270555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:19.270555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:19.270555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:19.270555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:19.270555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:19.270555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:19.270555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:19.270555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:19.270555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:19.270555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:19.270555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:19.270555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:19.270555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:19.270555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:19.270555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:19.271463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:19.271463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:19.271463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:19.271463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:19.273927 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 303/619] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:19.282622 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b021e09-7ed9-4767-957b-b79eeedcf0a4 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:19.285836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:19.338955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b67c6-85ce-43ef-9d9e-516bde214fec None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:19.378029 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64067ab2-458a-4700-a951-8c91b8ec3b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:19.379074 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64067ab2-458a-4700-a951-8c91b8ec3b48 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:19.379504 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64067ab2-458a-4700-a951-8c91b8ec3b48 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:19.379941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64067ab2-458a-4700-a951-8c91b8ec3b48 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:19.380943 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64067ab2-458a-4700-a951-8c91b8ec3b48 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:19.381438 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64067ab2-458a-4700-a951-8c91b8ec3b48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:19.383159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64067ab2-458a-4700-a951-8c91b8ec3b48 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:19.403093 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 317/620] 10.222.0.71 () {56 vars in 1135 bytes} [Thu Jul 21 20:26:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:19.409699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac57d3e-91f1-4d18-979f-e7e52d24fc03 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:19.410252 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac57d3e-91f1-4d18-979f-e7e52d24fc03 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:19.410707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac57d3e-91f1-4d18-979f-e7e52d24fc03 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:19.412574 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 318/621] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:19.413512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac57d3e-91f1-4d18-979f-e7e52d24fc03 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:19.490461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b021e09-7ed9-4767-957b-b79eeedcf0a4 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:19.538457 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac57d3e-91f1-4d18-979f-e7e52d24fc03 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:19.551285 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:19.561185 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59df0d-988f-4038-a61a-13e25ec4e27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:19.562549 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59df0d-988f-4038-a61a-13e25ec4e27c None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:19.562929 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59df0d-988f-4038-a61a-13e25ec4e27c None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:19.563424 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59df0d-988f-4038-a61a-13e25ec4e27c None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:19.564845 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59df0d-988f-4038-a61a-13e25ec4e27c None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:19.565425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59df0d-988f-4038-a61a-13e25ec4e27c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:19.568185 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59df0d-988f-4038-a61a-13e25ec4e27c None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:19.581873 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a43785-ea4a-4550-9c5f-39351c2b1259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:19.582429 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:19.584134 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a43785-ea4a-4550-9c5f-39351c2b1259 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:19.584570 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a43785-ea4a-4550-9c5f-39351c2b1259 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:19.584986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a43785-ea4a-4550-9c5f-39351c2b1259 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:19.597253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 304/622] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:19.598116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a43785-ea4a-4550-9c5f-39351c2b1259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:19.599651 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9659a4-cfda-4be9-bb75-a49f72871dcb None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:19.600253 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a43785-ea4a-4550-9c5f-39351c2b1259 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:19.600360 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9659a4-cfda-4be9-bb75-a49f72871dcb None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:19.600849 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9659a4-cfda-4be9-bb75-a49f72871dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:19.619857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-d1a43785-ea4a-4550-9c5f-39351c2b1259 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:19.619857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:19.619857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:19.619857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:19.619857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:19.619857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:19.619857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:19.619857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:19.619857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:19.619857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:19.619857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:19.619857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:19.619857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:19.619857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:19.619857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:19.620839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:19.620839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:19.620839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:19.620839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:19.620839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:19.620839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:19.620839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:19.620839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:19.620839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:19.620839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:19.620839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:19.620839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:19.620839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:19.620839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:19.620839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:19.620839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:19.620839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:19.621789 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:19.621789 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:19.621789 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:19.621789 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:19.624345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 319/623] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:19.630378 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a43785-ea4a-4550-9c5f-39351c2b1259 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:19.633047 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-4a9659a4-cfda-4be9-bb75-a49f72871dcb None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:19.635339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-4a9659a4-cfda-4be9-bb75-a49f72871dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:19.753465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a43785-ea4a-4550-9c5f-39351c2b1259 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:19.767265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 305/624] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:19.768892 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9659a4-cfda-4be9-bb75-a49f72871dcb None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:19.796900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00dc8a-479a-474a-817e-50456cb565ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:19.798313 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00dc8a-479a-474a-817e-50456cb565ae None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:19.798906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00dc8a-479a-474a-817e-50456cb565ae None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:19.799380 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00dc8a-479a-474a-817e-50456cb565ae None admin] PATH_INFO: `/v3/users/nova` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:19.809123 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00dc8a-479a-474a-817e-50456cb565ae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:19.811380 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00dc8a-479a-474a-817e-50456cb565ae None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:19.832275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-4d00dc8a-479a-474a-817e-50456cb565ae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:26:19.832275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:19.832275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:19.832275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:19.832275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:19.832275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:19.832275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:19.832275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:19.832275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:19.832275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:19.832275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:19.832275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:19.832275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:19.832275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:19.832275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:19.832275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:19.833337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:19.833337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:19.833337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:19.833337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:19.833337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:19.833337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:19.833337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:19.833337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:19.833337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:19.833337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:19.833337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:19.833337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:19.833337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:19.833337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:19.833337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:19.833337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:19.833337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:19.833337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:19.834325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:19.834325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:19.834325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:19.834325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:19.834325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:19.834325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:19.834325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:19.834325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:19.834325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:26:19.834325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:19.834859 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 320/625] 10.222.0.71 () {56 vars in 1093 bytes} [Thu Jul 21 20:26:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:19.836330 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00dc8a-479a-474a-817e-50456cb565ae None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:19.869552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:19.920877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9659a4-cfda-4be9-bb75-a49f72871dcb None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:19.958770 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:19.975507 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92429955-f665-48f6-9ca6-73596e803611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:19.976856 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92429955-f665-48f6-9ca6-73596e803611 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:19.977695 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92429955-f665-48f6-9ca6-73596e803611 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:19.978262 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92429955-f665-48f6-9ca6-73596e803611 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:19.979924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92429955-f665-48f6-9ca6-73596e803611 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:19.980587 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92429955-f665-48f6-9ca6-73596e803611 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:19.983203 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92429955-f665-48f6-9ca6-73596e803611 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:19.991921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00dc8a-479a-474a-817e-50456cb565ae None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:20.007589 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 306/626] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:20.009053 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92429955-f665-48f6-9ca6-73596e803611 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:20.051525 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6cce23-3b97-4368-94ff-2ae205078906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:20.053194 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cce23-3b97-4368-94ff-2ae205078906 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:20.053693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cce23-3b97-4368-94ff-2ae205078906 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:20.054146 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cce23-3b97-4368-94ff-2ae205078906 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:20.070331 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6cce23-3b97-4368-94ff-2ae205078906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:20.081021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-fd6cce23-3b97-4368-94ff-2ae205078906 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:20.081021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:20.081021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:20.081021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:20.081021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:20.081021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:20.081021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:20.081021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:20.081021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:20.081021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:20.081021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:20.081021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:20.081021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:20.081021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:20.081021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:20.081021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:20.082051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:20.082051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:20.082051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:20.082051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:20.082051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:20.082051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:20.082051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:20.082051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:20.082051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:20.082051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:20.082051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:20.082051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:20.082051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:20.082051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:20.082051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:20.082051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:20.082897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:20.082897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:20.082897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:20.082897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:20.082897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:20.082897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:20.082897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:20.082897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:20.082897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:20.082897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:20.082897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:20.082897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:20.086126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 321/627] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:20.087467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6cce23-3b97-4368-94ff-2ae205078906 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:20.169509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92429955-f665-48f6-9ca6-73596e803611 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:20.216213 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf51327-2185-4f38-b12c-de92e87d12ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:20.219287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf51327-2185-4f38-b12c-de92e87d12ff None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:20.219800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf51327-2185-4f38-b12c-de92e87d12ff None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:20.220304 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf51327-2185-4f38-b12c-de92e87d12ff None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:20.222293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf51327-2185-4f38-b12c-de92e87d12ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:20.222862 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf51327-2185-4f38-b12c-de92e87d12ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:20.224545 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:20.225875 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf51327-2185-4f38-b12c-de92e87d12ff None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:20.251208 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 307/628] 10.222.0.71 () {56 vars in 1133 bytes} [Thu Jul 21 20:26:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:20.251739 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6cce23-3b97-4368-94ff-2ae205078906 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:20.252218 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf51327-2185-4f38-b12c-de92e87d12ff None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:20.297230 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79316a7c-4ba3-432b-b734-eb9c86a41a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:20.298146 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79316a7c-4ba3-432b-b734-eb9c86a41a35 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:20.298616 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79316a7c-4ba3-432b-b734-eb9c86a41a35 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:20.298925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79316a7c-4ba3-432b-b734-eb9c86a41a35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:20.306965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79316a7c-4ba3-432b-b734-eb9c86a41a35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:20.308575 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79316a7c-4ba3-432b-b734-eb9c86a41a35 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:20.313538 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:20.325947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-79316a7c-4ba3-432b-b734-eb9c86a41a35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:20.325947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:20.325947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:20.325947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:20.325947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:20.325947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:20.325947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:20.325947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:20.325947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:20.325947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:20.325947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:20.325947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:20.325947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:20.325947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:20.325947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:20.335413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:20.335413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:20.335413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:20.335413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:20.335413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:20.335413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:20.335413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:20.335413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:20.335413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:20.335413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:20.335413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:20.335413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:20.335413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:20.335413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:20.335413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:20.335413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:20.335413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:20.337529 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:20.337529 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:20.337529 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:20.337529 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:20.337529 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:20.337529 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:20.337529 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:20.337529 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:20.337529 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 322/629] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:20.337529 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79316a7c-4ba3-432b-b734-eb9c86a41a35 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:20.397014 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf51327-2185-4f38-b12c-de92e87d12ff None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:20.439125 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c88538-7bea-4eb8-9f6b-c00e8b3786ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:20.439795 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c88538-7bea-4eb8-9f6b-c00e8b3786ab None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:20.440073 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c88538-7bea-4eb8-9f6b-c00e8b3786ab None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:20.440343 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c88538-7bea-4eb8-9f6b-c00e8b3786ab None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:20.441111 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c88538-7bea-4eb8-9f6b-c00e8b3786ab None admin] RBAC: Adding query filter params (name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:20.441421 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c88538-7bea-4eb8-9f6b-c00e8b3786ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:20.442823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c88538-7bea-4eb8-9f6b-c00e8b3786ab None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:20.458041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 308/630] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:26:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:20.460399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19886d87-e291-4a19-b536-d84e4559da93 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:20.460904 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19886d87-e291-4a19-b536-d84e4559da93 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:20.461507 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19886d87-e291-4a19-b536-d84e4559da93 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:20.463950 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 309/631] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:20.465828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19886d87-e291-4a19-b536-d84e4559da93 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:20.482124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79316a7c-4ba3-432b-b734-eb9c86a41a35 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:20.539451 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e9e2f-ecd5-4f5f-aa07-37850f1e485a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:20.540685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9e2f-ecd5-4f5f-aa07-37850f1e485a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:20.541118 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9e2f-ecd5-4f5f-aa07-37850f1e485a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:20.541653 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9e2f-ecd5-4f5f-aa07-37850f1e485a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:20.564198 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:20.629590 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19886d87-e291-4a19-b536-d84e4559da93 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:20.637351 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e9e2f-ecd5-4f5f-aa07-37850f1e485a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:20.638850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e9e2f-ecd5-4f5f-aa07-37850f1e485a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:20.641812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:20.674026 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcf261-e4b9-4710-a66e-6efcf317fb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:20.674722 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcf261-e4b9-4710-a66e-6efcf317fb58 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:20.675004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcf261-e4b9-4710-a66e-6efcf317fb58 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:20.675283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcf261-e4b9-4710-a66e-6efcf317fb58 None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:20.676110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fcf261-e4b9-4710-a66e-6efcf317fb58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:20.676425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fcf261-e4b9-4710-a66e-6efcf317fb58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:20.677927 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fcf261-e4b9-4710-a66e-6efcf317fb58 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:20.694127 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 310/632] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:20.695155 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcf261-e4b9-4710-a66e-6efcf317fb58 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:20.810313 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 323/633] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:26:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:26:20.813009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bad9a-49b1-4984-9349-6fe61924e82c None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:20.813633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bad9a-49b1-4984-9349-6fe61924e82c None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:20.814141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bad9a-49b1-4984-9349-6fe61924e82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:20.817982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcf261-e4b9-4710-a66e-6efcf317fb58 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:20.834516 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-7c2bad9a-49b1-4984-9349-6fe61924e82c None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:20.835722 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-7c2bad9a-49b1-4984-9349-6fe61924e82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:20.859892 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309fdce-6905-4e2c-9130-9557529e03c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:20.860781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309fdce-6905-4e2c-9130-9557529e03c3 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:20.861087 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309fdce-6905-4e2c-9130-9557529e03c3 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:20.861395 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309fdce-6905-4e2c-9130-9557529e03c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:20.872754 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309fdce-6905-4e2c-9130-9557529e03c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:20.874263 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309fdce-6905-4e2c-9130-9557529e03c3 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:20.879237 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:20.893357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-5309fdce-6905-4e2c-9130-9557529e03c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:20.893357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:20.893357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:20.893357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:20.893357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:20.893357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:20.893357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:20.893357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:20.893357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:20.893357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:20.893357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:20.893357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:20.893357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:20.893357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:20.893357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:20.894517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:20.894517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:20.894517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:20.894517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:20.894517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:20.894517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:20.894517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:20.894517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:20.894517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:20.894517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:20.894517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:20.894517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:20.894517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:20.894517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:20.894517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:20.894517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:20.894517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:20.900504 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:20.900504 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:20.900504 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:20.900504 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:20.900504 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 311/634] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:20.900504 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309fdce-6905-4e2c-9130-9557529e03c3 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:20.920491 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 324/635] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:20.921368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bad9a-49b1-4984-9349-6fe61924e82c None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:20.935388 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:21.009986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309fdce-6905-4e2c-9130-9557529e03c3 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:21.038237 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bad9a-49b1-4984-9349-6fe61924e82c None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:21.047041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa3b1d-725f-4ba0-a7b7-784f03e89827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:21.047946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa3b1d-725f-4ba0-a7b7-784f03e89827 None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:21.048211 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa3b1d-725f-4ba0-a7b7-784f03e89827 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:21.048484 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa3b1d-725f-4ba0-a7b7-784f03e89827 None admin] PATH_INFO: `/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/3babb078203647e2b3ef3b2787b84e3b/roles/9a48ab1bf32a45828e9e73599c980dac` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:21.064702 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa3b1d-725f-4ba0-a7b7-784f03e89827 None admin] RBAC: Authorizing `identity:create_grant(project_id=37e4afa595894a9994a8e6981ebb7c41, user_id=3babb078203647e2b3ef3b2787b84e3b, role_id=9a48ab1bf32a45828e9e73599c980dac)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:21.066195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa3b1d-725f-4ba0-a7b7-784f03e89827 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:21.074452 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8d289-fff6-4a73-90e8-24c7a188ed4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:21.075790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8d289-fff6-4a73-90e8-24c7a188ed4e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:21.076086 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8d289-fff6-4a73-90e8-24c7a188ed4e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:21.076361 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8d289-fff6-4a73-90e8-24c7a188ed4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:21.081369 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 312/636] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:26:20 2022] PUT /identity/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/3babb078203647e2b3ef3b2787b84e3b/roles/9a48ab1bf32a45828e9e73599c980dac => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:26:21.082789 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abd352-4a8b-4d8b-b28a-972d7a415e57 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:21.083136 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abd352-4a8b-4d8b-b28a-972d7a415e57 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:21.083423 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abd352-4a8b-4d8b-b28a-972d7a415e57 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:21.085109 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 313/637] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:21.085921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abd352-4a8b-4d8b-b28a-972d7a415e57 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:21.093256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8d289-fff6-4a73-90e8-24c7a188ed4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:21.095288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8d289-fff6-4a73-90e8-24c7a188ed4e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:21.115675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-aac8d289-fff6-4a73-90e8-24c7a188ed4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:21.115675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:21.115675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:21.115675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:21.115675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:21.115675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:21.115675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:21.115675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:21.115675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:21.115675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:21.115675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:21.115675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:21.115675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:21.115675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:21.115675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:21.117151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:21.117151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:21.117151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:21.117151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:21.117151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:21.117151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:21.117151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:21.117151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:21.117151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:21.117151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:21.117151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:21.117151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:21.117151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:21.117151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:21.117151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:21.117151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:21.117151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:21.118253 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:21.118253 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:21.118253 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:21.118253 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:21.121933 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 325/638] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:21.122898 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8d289-fff6-4a73-90e8-24c7a188ed4e None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:21.164848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:21.207118 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:21.250934 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abd352-4a8b-4d8b-b28a-972d7a415e57 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:21.263557 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8d289-fff6-4a73-90e8-24c7a188ed4e None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:21.291187 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02837776-7d00-47eb-87b1-519f2bd38a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:21.291861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02837776-7d00-47eb-87b1-519f2bd38a01 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:21.292253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02837776-7d00-47eb-87b1-519f2bd38a01 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:21.292883 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02837776-7d00-47eb-87b1-519f2bd38a01 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:21.293295 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02837776-7d00-47eb-87b1-519f2bd38a01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:21.293611 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02837776-7d00-47eb-87b1-519f2bd38a01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:21.295255 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02837776-7d00-47eb-87b1-519f2bd38a01 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:21.313791 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d1df2-1cab-4d8f-ab20-cf0de6040dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:21.314694 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d1df2-1cab-4d8f-ab20-cf0de6040dc1 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:21.315142 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d1df2-1cab-4d8f-ab20-cf0de6040dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:21.315410 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d1df2-1cab-4d8f-ab20-cf0de6040dc1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:21.317238 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 314/639] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:21.318955 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52127d0e-6611-4fb8-acd2-fc7bced6b4bb None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:21.319249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52127d0e-6611-4fb8-acd2-fc7bced6b4bb None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:21.319530 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52127d0e-6611-4fb8-acd2-fc7bced6b4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:21.332452 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d1df2-1cab-4d8f-ab20-cf0de6040dc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:21.340969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-7b8d1df2-1cab-4d8f-ab20-cf0de6040dc1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:21.340969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:21.340969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:21.340969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:21.340969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:21.340969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:21.340969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:21.340969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:21.340969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:21.340969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:21.340969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:21.340969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:21.340969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:21.340969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:21.340969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:21.340969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:21.342816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:21.342816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:21.342816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:21.342816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:21.342816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:21.342816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:21.342816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:21.342816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:21.342816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:21.342816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:21.342816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:21.342816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:21.342816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:21.342816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:21.342816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:21.342816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:21.343707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:21.343707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:21.343707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:21.343707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:21.343707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:21.343707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:21.343707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:21.343707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:21.343707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:21.343707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:21.343707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:21.343707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:21.344396 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-52127d0e-6611-4fb8-acd2-fc7bced6b4bb None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:21.344396 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-52127d0e-6611-4fb8-acd2-fc7bced6b4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:21.347578 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 326/640] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:21.348474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d1df2-1cab-4d8f-ab20-cf0de6040dc1 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:21.430928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 315/641] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:21.432136 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52127d0e-6611-4fb8-acd2-fc7bced6b4bb None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:21.477283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:21.502422 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d1df2-1cab-4d8f-ab20-cf0de6040dc1 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:21.530266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:21.552176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9a0d1-8d68-447d-a82b-b0b3dbb27eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:21.553191 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9a0d1-8d68-447d-a82b-b0b3dbb27eea None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:21.553191 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9a0d1-8d68-447d-a82b-b0b3dbb27eea None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:21.553421 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9a0d1-8d68-447d-a82b-b0b3dbb27eea None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:21.554161 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9a0d1-8d68-447d-a82b-b0b3dbb27eea None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:21.554469 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9a0d1-8d68-447d-a82b-b0b3dbb27eea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:21.555865 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9a0d1-8d68-447d-a82b-b0b3dbb27eea None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:21.571222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 327/642] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:21.572432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9a0d1-8d68-447d-a82b-b0b3dbb27eea None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:21.586523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52127d0e-6611-4fb8-acd2-fc7bced6b4bb None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:21.628811 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ded509-d5fe-4ced-b20a-e7123a99adaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:21.629661 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ded509-d5fe-4ced-b20a-e7123a99adaa None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:21.629949 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ded509-d5fe-4ced-b20a-e7123a99adaa None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:21.630257 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ded509-d5fe-4ced-b20a-e7123a99adaa None admin] PATH_INFO: `/v3/users/glance` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:21.639686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ded509-d5fe-4ced-b20a-e7123a99adaa None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:21.641189 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ded509-d5fe-4ced-b20a-e7123a99adaa None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:21.656642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-73ded509-d5fe-4ced-b20a-e7123a99adaa None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 20:26:21.656642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:21.656642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:21.656642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:21.656642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:21.656642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:21.656642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:21.656642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:21.656642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:21.656642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:21.656642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:21.656642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:21.656642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:21.656642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:21.656642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:21.656642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:21.657919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:21.657919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:21.657919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:21.657919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:21.657919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:21.657919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:21.657919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:21.657919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:21.657919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:21.657919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:21.657919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:21.657919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:21.657919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:21.657919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:21.657919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:21.657919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:21.657919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:21.657919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:21.658986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:21.658986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:21.658986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:21.658986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:21.658986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:21.658986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:21.658986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:21.658986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:21.658986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 20:26:21.658986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:21.658986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 316/643] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:26:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:21.659682 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ded509-d5fe-4ced-b20a-e7123a99adaa None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:21.702686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9a0d1-8d68-447d-a82b-b0b3dbb27eea None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:21.739578 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4583f4a-e5cd-450f-82e5-c649c343edf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:21.740352 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4583f4a-e5cd-450f-82e5-c649c343edf8 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:21.740654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4583f4a-e5cd-450f-82e5-c649c343edf8 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:21.740964 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4583f4a-e5cd-450f-82e5-c649c343edf8 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:21.741841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4583f4a-e5cd-450f-82e5-c649c343edf8 None admin] RBAC: Adding query filter params (name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:21.742153 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4583f4a-e5cd-450f-82e5-c649c343edf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:21.743624 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4583f4a-e5cd-450f-82e5-c649c343edf8 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:21.757502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 328/644] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:26:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:21.757502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4583f4a-e5cd-450f-82e5-c649c343edf8 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:21.768107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:21.785301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ded509-d5fe-4ced-b20a-e7123a99adaa None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:21.808131 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:21.827743 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6add1ae-e93e-4d58-827b-f0b850f99acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:21.828465 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6add1ae-e93e-4d58-827b-f0b850f99acb None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:21.828740 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6add1ae-e93e-4d58-827b-f0b850f99acb None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:21.829010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6add1ae-e93e-4d58-827b-f0b850f99acb None admin] PATH_INFO: `/v3/roles/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:21.840521 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6add1ae-e93e-4d58-827b-f0b850f99acb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:21.848327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-d6add1ae-e93e-4d58-827b-f0b850f99acb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:21.848327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:21.848327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:21.848327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:21.848327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:21.848327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:21.848327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:21.848327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:21.848327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:21.848327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:21.848327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:21.848327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:21.848327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:21.848327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:21.848327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:21.848327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:21.849470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:21.849470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:21.849470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:21.849470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:21.849470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:21.849470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:21.849470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:21.849470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:21.849470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:21.849470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:21.849470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:21.849470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:21.849470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:21.849470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:21.849470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:21.849470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:21.856399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:21.856399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:21.856399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:21.856399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:21.856399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:21.856399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:21.856399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:21.856399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:21.856399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:21.856399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:21.856399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:21.856399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:21.856399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 317/645] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:21.856399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6add1ae-e93e-4d58-827b-f0b850f99acb None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:21.880141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4583f4a-e5cd-450f-82e5-c649c343edf8 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:21.917689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa536982-692d-4a9e-b7ec-6bfd1180a487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:21.918522 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa536982-692d-4a9e-b7ec-6bfd1180a487 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:21.918988 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa536982-692d-4a9e-b7ec-6bfd1180a487 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:21.919735 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa536982-692d-4a9e-b7ec-6bfd1180a487 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:21.927724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa536982-692d-4a9e-b7ec-6bfd1180a487 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:21.929238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa536982-692d-4a9e-b7ec-6bfd1180a487 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:21.946079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-fa536982-692d-4a9e-b7ec-6bfd1180a487 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:21.946079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:21.946079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:21.946079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:21.946079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:21.946079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:21.946079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:21.946079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:21.946079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:21.946079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:21.946079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:21.946079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:21.946079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:21.946079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:21.946079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:21.947342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:21.947342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:21.947342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:21.947342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:21.947342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:21.947342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:21.947342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:21.947342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:21.947342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:21.947342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:21.947342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:21.947342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:21.947342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:21.947342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:21.947342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:21.947342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:21.947342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:21.948754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:21.948754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:21.948754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:21.948754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:21.948754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:21.948754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:21.948754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:21.948754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:21.952217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 329/646] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:21.953105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa536982-692d-4a9e-b7ec-6bfd1180a487 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:21.979387 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6add1ae-e93e-4d58-827b-f0b850f99acb None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:22.019320 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1d1a5-3ed4-46dc-bab5-80c2cd6d2e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:22.020079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1d1a5-3ed4-46dc-bab5-80c2cd6d2e70 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:22.020381 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1d1a5-3ed4-46dc-bab5-80c2cd6d2e70 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:22.020662 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1d1a5-3ed4-46dc-bab5-80c2cd6d2e70 None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:22.021673 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1d1a5-3ed4-46dc-bab5-80c2cd6d2e70 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:22.021992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1d1a5-3ed4-46dc-bab5-80c2cd6d2e70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:22.023450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1d1a5-3ed4-46dc-bab5-80c2cd6d2e70 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:22.028000 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:22.046664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 318/647] 10.222.0.71 () {56 vars in 1137 bytes} [Thu Jul 21 20:26:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:22.051774 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1d1a5-3ed4-46dc-bab5-80c2cd6d2e70 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:22.074646 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa536982-692d-4a9e-b7ec-6bfd1180a487 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:22.110722 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:22.118284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88167d4-d1eb-4cfa-afe4-3d0b773f17eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:22.120256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88167d4-d1eb-4cfa-afe4-3d0b773f17eb None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:22.120947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88167d4-d1eb-4cfa-afe4-3d0b773f17eb None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:22.121720 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88167d4-d1eb-4cfa-afe4-3d0b773f17eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:22.139665 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88167d4-d1eb-4cfa-afe4-3d0b773f17eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:22.142223 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88167d4-d1eb-4cfa-afe4-3d0b773f17eb None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:22.162102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-c88167d4-d1eb-4cfa-afe4-3d0b773f17eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:22.162102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:22.162102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:22.162102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:22.162102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:22.162102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:22.162102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:22.162102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:22.162102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:22.162102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:22.162102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:22.162102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:22.162102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:22.162102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:22.162102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:22.163089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:22.163089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:22.163089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:22.163089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:22.163089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:22.163089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:22.163089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:22.163089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:22.163089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:22.163089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:22.163089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:22.163089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:22.163089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:22.163089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:22.163089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:22.163089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:22.163089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:22.163989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:22.163989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:22.163989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:22.163989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:22.163989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 330/648] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:22.164279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88167d4-d1eb-4cfa-afe4-3d0b773f17eb None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:22.182233 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1d1a5-3ed4-46dc-bab5-80c2cd6d2e70 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:22.219824 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc14b0-cd61-4c2d-9091-3c455df1c03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:22.220492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc14b0-cd61-4c2d-9091-3c455df1c03c None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:22.220771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc14b0-cd61-4c2d-9091-3c455df1c03c None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:22.221044 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc14b0-cd61-4c2d-9091-3c455df1c03c None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:22.221830 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc14b0-cd61-4c2d-9091-3c455df1c03c None admin] RBAC: Adding query filter params (name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:22.222136 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc14b0-cd61-4c2d-9091-3c455df1c03c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:22.223520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc14b0-cd61-4c2d-9091-3c455df1c03c None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:22.237130 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 319/649] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:26:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:22.237130 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc14b0-cd61-4c2d-9091-3c455df1c03c None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:22.295364 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88167d4-d1eb-4cfa-afe4-3d0b773f17eb None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:22.319834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:22.345095 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55500498-6d8d-4013-a8cc-670f56c7d17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:22.346205 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55500498-6d8d-4013-a8cc-670f56c7d17b None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:22.346525 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55500498-6d8d-4013-a8cc-670f56c7d17b None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:22.347030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55500498-6d8d-4013-a8cc-670f56c7d17b None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:22.348573 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55500498-6d8d-4013-a8cc-670f56c7d17b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:22.349174 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55500498-6d8d-4013-a8cc-670f56c7d17b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:22.352015 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55500498-6d8d-4013-a8cc-670f56c7d17b None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:22.352791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:22.356177 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc14b0-cd61-4c2d-9091-3c455df1c03c None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:22.374681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 331/650] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:22.375925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55500498-6d8d-4013-a8cc-670f56c7d17b None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:22.400673 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304a375-5ad6-49e3-8741-07a7c35b378d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:22.401458 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304a375-5ad6-49e3-8741-07a7c35b378d None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:22.401746 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304a375-5ad6-49e3-8741-07a7c35b378d None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:22.402028 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304a375-5ad6-49e3-8741-07a7c35b378d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:22.494818 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304a375-5ad6-49e3-8741-07a7c35b378d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:22.498450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304a375-5ad6-49e3-8741-07a7c35b378d None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:22.524863 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55500498-6d8d-4013-a8cc-670f56c7d17b None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:22.581086 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f29f29-4007-4be0-9011-b3b066550ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:22.581787 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f29f29-4007-4be0-9011-b3b066550ad7 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:22.582047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f29f29-4007-4be0-9011-b3b066550ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:22.582328 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f29f29-4007-4be0-9011-b3b066550ad7 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:22.583062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f29f29-4007-4be0-9011-b3b066550ad7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:22.583376 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f29f29-4007-4be0-9011-b3b066550ad7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:22.584759 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f29f29-4007-4be0-9011-b3b066550ad7 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:22.602465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 332/651] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:22.602465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f29f29-4007-4be0-9011-b3b066550ad7 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:22.609365 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:22.614407 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 320/652] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:26:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:26:22.620305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304a375-5ad6-49e3-8741-07a7c35b378d None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:22.643850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:22.720063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f29f29-4007-4be0-9011-b3b066550ad7 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:22.746785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304a375-5ad6-49e3-8741-07a7c35b378d None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:22.760420 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f2a2f-3eee-4f2c-9e8c-7a083bcd73b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:22.761352 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f2a2f-3eee-4f2c-9e8c-7a083bcd73b4 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:22.761702 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f2a2f-3eee-4f2c-9e8c-7a083bcd73b4 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:22.762051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f2a2f-3eee-4f2c-9e8c-7a083bcd73b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:22.773152 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f2a2f-3eee-4f2c-9e8c-7a083bcd73b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:22.774740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f2a2f-3eee-4f2c-9e8c-7a083bcd73b4 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:22.785986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fb355-09ae-46b0-a613-a973ce17523d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:22.786655 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fb355-09ae-46b0-a613-a973ce17523d None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:22.786942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fb355-09ae-46b0-a613-a973ce17523d None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:22.787221 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fb355-09ae-46b0-a613-a973ce17523d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:22.788152 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fb355-09ae-46b0-a613-a973ce17523d None admin] RBAC: Adding query filter params (role.id=9a48ab1bf32a45828e9e73599c980dac, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=6fbc5240f5214aaea4418052b94739e2) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:22.788469 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fb355-09ae-46b0-a613-a973ce17523d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:22.789954 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fb355-09ae-46b0-a613-a973ce17523d None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:22.793620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-120f2a2f-3eee-4f2c-9e8c-7a083bcd73b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:22.793620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:22.793620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:22.793620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:22.793620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:22.793620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:22.793620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:22.793620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:22.793620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:22.793620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:22.793620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:22.793620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:22.793620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:22.793620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:22.793620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:22.795249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:22.795249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:22.795249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:22.795249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:22.795249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:22.795249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:22.795249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:22.795249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:22.795249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:22.795249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:22.795249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:22.795249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:22.795249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:22.795249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:22.795249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:22.795249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:22.795249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:22.796426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:22.796426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:22.796426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:22.796426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:22.799852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 321/653] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:26:22 2022] GET /identity/v3/role_assignments?user.id=6fbc5240f5214aaea4418052b94739e2&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=9a48ab1bf32a45828e9e73599c980dac => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:22.800212 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 333/654] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:22.801047 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fb355-09ae-46b0-a613-a973ce17523d None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:22.801184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f2a2f-3eee-4f2c-9e8c-7a083bcd73b4 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:22.892188 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:22.942028 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:22.952369 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fb355-09ae-46b0-a613-a973ce17523d None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:22.956742 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f2a2f-3eee-4f2c-9e8c-7a083bcd73b4 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:23.003287 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8c0c0-a012-4b5c-bc48-28913af6832c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:23.004214 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfb8b7-0fd5-4ff6-8e06-5eeba89fe65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:23.004694 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8c0c0-a012-4b5c-bc48-28913af6832c None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:23.004694 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8c0c0-a012-4b5c-bc48-28913af6832c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:23.005020 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfb8b7-0fd5-4ff6-8e06-5eeba89fe65d None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:23.005169 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8c0c0-a012-4b5c-bc48-28913af6832c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:23.005326 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfb8b7-0fd5-4ff6-8e06-5eeba89fe65d None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:23.005600 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfb8b7-0fd5-4ff6-8e06-5eeba89fe65d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:23.016375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbfb8b7-0fd5-4ff6-8e06-5eeba89fe65d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:23.016375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbfb8b7-0fd5-4ff6-8e06-5eeba89fe65d None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:23.025907 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8c0c0-a012-4b5c-bc48-28913af6832c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:23.027394 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8c0c0-a012-4b5c-bc48-28913af6832c None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:23.036704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-5cbfb8b7-0fd5-4ff6-8e06-5eeba89fe65d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 20:26:23.036704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:23.036704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:23.036704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:23.036704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:23.036704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:23.036704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:23.036704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:23.036704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:23.036704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:23.036704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:23.036704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:23.036704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:23.036704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:23.036704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:23.036704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:23.037921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:23.037921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:23.037921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:23.037921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:23.037921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:23.037921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:23.037921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:23.037921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:23.037921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:23.037921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:23.037921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:23.037921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:23.037921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:23.037921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:23.037921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:23.037921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:23.037921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:23.037921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:23.039055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:23.039055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:23.039055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:23.039055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:23.039055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:23.039055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:23.039055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:23.039055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:23.039055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 20:26:23.039055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:23.040257 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 322/655] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:26:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:23.040928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfb8b7-0fd5-4ff6-8e06-5eeba89fe65d None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:23.053842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-7ab8c0c0-a012-4b5c-bc48-28913af6832c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:23.053842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:23.053842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:23.053842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:23.053842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:23.053842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:23.053842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:23.053842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:23.053842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:23.053842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:23.053842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:23.053842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:23.053842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:23.053842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:23.053842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:23.055172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:23.055172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:23.055172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:23.055172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:23.055172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:23.055172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:23.055172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:23.055172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:23.055172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:23.055172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:23.055172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:23.055172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:23.055172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:23.055172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:23.055172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:23.055172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:23.055172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:23.056620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:23.056620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:23.056620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:23.056620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:23.057631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 334/656] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:23.058608 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8c0c0-a012-4b5c-bc48-28913af6832c None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:23.176500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfb8b7-0fd5-4ff6-8e06-5eeba89fe65d None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:23.191794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8c0c0-a012-4b5c-bc48-28913af6832c None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:23.205548 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:23.216287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c991200-2862-43dd-9089-8c54bfad2468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:23.216986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c991200-2862-43dd-9089-8c54bfad2468 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:23.217303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c991200-2862-43dd-9089-8c54bfad2468 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:23.217579 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c991200-2862-43dd-9089-8c54bfad2468 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:23.218383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c991200-2862-43dd-9089-8c54bfad2468 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:23.218696 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c991200-2862-43dd-9089-8c54bfad2468 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:23.220298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c991200-2862-43dd-9089-8c54bfad2468 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:23.235829 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 323/657] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:23.236921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c991200-2862-43dd-9089-8c54bfad2468 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:23.245418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedae02-9723-4f29-81fb-3bc0e822e5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:23.246186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedae02-9723-4f29-81fb-3bc0e822e5b2 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:23.246552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedae02-9723-4f29-81fb-3bc0e822e5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:23.246982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedae02-9723-4f29-81fb-3bc0e822e5b2 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:23.247960 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feedae02-9723-4f29-81fb-3bc0e822e5b2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:23.248342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feedae02-9723-4f29-81fb-3bc0e822e5b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:23.250005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feedae02-9723-4f29-81fb-3bc0e822e5b2 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:23.268461 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:23.273283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 335/658] 10.222.0.71 () {56 vars in 1137 bytes} [Thu Jul 21 20:26:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:23.274310 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedae02-9723-4f29-81fb-3bc0e822e5b2 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:23.358723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c991200-2862-43dd-9089-8c54bfad2468 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:23.388973 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedae02-9723-4f29-81fb-3bc0e822e5b2 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:23.399540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f214f-6d2f-4954-afd7-ccd96510d79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:23.400231 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f214f-6d2f-4954-afd7-ccd96510d79f None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:23.400518 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f214f-6d2f-4954-afd7-ccd96510d79f None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:23.400799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f214f-6d2f-4954-afd7-ccd96510d79f None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:23.401574 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f214f-6d2f-4954-afd7-ccd96510d79f None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:23.401883 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f214f-6d2f-4954-afd7-ccd96510d79f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:23.403349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f214f-6d2f-4954-afd7-ccd96510d79f None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:23.421557 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 324/659] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:23.421557 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9291da7-1d0b-45b9-b079-a931cb3b1111 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:23.421557 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9291da7-1d0b-45b9-b079-a931cb3b1111 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:23.421557 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9291da7-1d0b-45b9-b079-a931cb3b1111 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:23.421557 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 325/660] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:23.422037 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9291da7-1d0b-45b9-b079-a931cb3b1111 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:23.434024 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a51cb5-91a4-4438-952d-311ede06e72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:23.434842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a51cb5-91a4-4438-952d-311ede06e72f None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:23.435129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a51cb5-91a4-4438-952d-311ede06e72f None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:23.435418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a51cb5-91a4-4438-952d-311ede06e72f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:23.444151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a51cb5-91a4-4438-952d-311ede06e72f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:23.445701 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a51cb5-91a4-4438-952d-311ede06e72f None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:23.462668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-b4a51cb5-91a4-4438-952d-311ede06e72f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 20:26:23.462668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:23.462668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:23.462668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:23.462668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:23.462668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:23.462668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:23.462668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:23.462668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:23.462668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:23.462668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:23.462668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:23.462668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:23.462668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:23.462668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:23.462668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:23.463689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:23.463689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:23.463689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:23.463689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:23.463689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:23.463689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:23.463689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:23.463689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:23.463689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:23.463689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:23.463689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:23.463689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:23.463689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:23.463689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:23.463689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:23.463689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:23.463689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:23.463689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:23.464644 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:23.464644 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:23.464644 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:23.464644 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:23.464644 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:23.464644 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:23.464644 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:23.464644 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:23.464644 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 20:26:23.464644 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:23.464644 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 336/661] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:23.464644 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a51cb5-91a4-4438-952d-311ede06e72f None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:23.505025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:23.548009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:23.570927 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9291da7-1d0b-45b9-b079-a931cb3b1111 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:23.599932 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a51cb5-91a4-4438-952d-311ede06e72f None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:23.612416 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaddb48-e5c5-477a-b005-cdf42b9ba040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:23.613101 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaddb48-e5c5-477a-b005-cdf42b9ba040 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:23.613389 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaddb48-e5c5-477a-b005-cdf42b9ba040 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:23.613668 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaddb48-e5c5-477a-b005-cdf42b9ba040 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:23.641183 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2963dcf-4f28-4a71-b4de-ad1d95d3d735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:23.642356 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2963dcf-4f28-4a71-b4de-ad1d95d3d735 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:23.642917 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2963dcf-4f28-4a71-b4de-ad1d95d3d735 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:23.643197 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2963dcf-4f28-4a71-b4de-ad1d95d3d735 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:23.654135 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2963dcf-4f28-4a71-b4de-ad1d95d3d735 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:23.655609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2963dcf-4f28-4a71-b4de-ad1d95d3d735 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:23.670965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-a2963dcf-4f28-4a71-b4de-ad1d95d3d735 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:23.670965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:23.670965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:23.670965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:23.670965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:23.670965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:23.670965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:23.670965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:23.670965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:23.670965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:23.670965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:23.670965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:23.670965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:23.670965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:23.670965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:23.672117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:23.672117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:23.672117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:23.672117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:23.672117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:23.672117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:23.672117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:23.672117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:23.672117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:23.672117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:23.672117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:23.672117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:23.672117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:23.672117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:23.672117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:23.672117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:23.672117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:23.673418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:23.673418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:23.673418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:23.673418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:23.673418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:23.673418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:23.673418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:23.673418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:23.676963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 337/662] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:23.678548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d3788-a4ea-4cc6-a8f8-c75d0f2af0a0 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:23.678663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d3788-a4ea-4cc6-a8f8-c75d0f2af0a0 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:23.678902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d3788-a4ea-4cc6-a8f8-c75d0f2af0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:23.692051 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaddb48-e5c5-477a-b005-cdf42b9ba040 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:23.693590 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaddb48-e5c5-477a-b005-cdf42b9ba040 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:23.698904 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-df8d3788-a4ea-4cc6-a8f8-c75d0f2af0a0 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:23.701248 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-df8d3788-a4ea-4cc6-a8f8-c75d0f2af0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:23.746528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:23.785133 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 338/663] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:23.785133 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d3788-a4ea-4cc6-a8f8-c75d0f2af0a0 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:23.832289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 326/664] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:26:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:26:23.833541 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaddb48-e5c5-477a-b005-cdf42b9ba040 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:23.846057 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:23.904522 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d3788-a4ea-4cc6-a8f8-c75d0f2af0a0 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:23.945404 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d04d0e-f27b-4c47-8d01-f7958fc9601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:23.946417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d04d0e-f27b-4c47-8d01-f7958fc9601f None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:23.946975 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d04d0e-f27b-4c47-8d01-f7958fc9601f None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:23.947249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d04d0e-f27b-4c47-8d01-f7958fc9601f None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:23.948119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d04d0e-f27b-4c47-8d01-f7958fc9601f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:23.948427 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d04d0e-f27b-4c47-8d01-f7958fc9601f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:23.948653 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaddb48-e5c5-477a-b005-cdf42b9ba040 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:23.949941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d04d0e-f27b-4c47-8d01-f7958fc9601f None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:23.980676 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 339/665] 10.222.0.71 () {56 vars in 1139 bytes} [Thu Jul 21 20:26:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:23.980676 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d04d0e-f27b-4c47-8d01-f7958fc9601f None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:23.996696 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e46a79-9d35-4c08-9b96-02bde2a291af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:23.997804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e46a79-9d35-4c08-9b96-02bde2a291af None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:23.998096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e46a79-9d35-4c08-9b96-02bde2a291af None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:23.998373 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e46a79-9d35-4c08-9b96-02bde2a291af None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:23.999214 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e46a79-9d35-4c08-9b96-02bde2a291af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:24.000053 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e46a79-9d35-4c08-9b96-02bde2a291af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:24.000976 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e46a79-9d35-4c08-9b96-02bde2a291af None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:24.019945 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 327/666] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:24.019945 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e46a79-9d35-4c08-9b96-02bde2a291af None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:24.026517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:24.120200 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d04d0e-f27b-4c47-8d01-f7958fc9601f None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:24.132221 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:24.164837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4572d-01a3-45f1-9036-26af65d9a6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:24.166032 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4572d-01a3-45f1-9036-26af65d9a6ff None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:24.166494 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4572d-01a3-45f1-9036-26af65d9a6ff None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:24.166918 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4572d-01a3-45f1-9036-26af65d9a6ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:24.167622 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e46a79-9d35-4c08-9b96-02bde2a291af None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:24.182394 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4572d-01a3-45f1-9036-26af65d9a6ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:24.191502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-59d4572d-01a3-45f1-9036-26af65d9a6ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:24.191502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:24.191502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:24.191502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:24.191502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:24.191502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:24.191502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:24.191502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:24.191502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:24.191502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:24.191502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:24.191502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:24.191502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:24.191502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:24.191502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:24.191502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:24.192949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:24.192949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:24.192949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:24.192949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:24.192949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:24.192949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:24.192949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:24.192949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:24.192949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:24.192949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:24.192949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:24.192949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:24.192949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:24.192949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:24.192949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:24.192949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:24.194304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:24.194304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:24.194304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:24.194304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:24.194304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:24.194304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:24.194304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:24.194304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:24.194304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:24.194304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:24.194304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:24.194304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:24.197867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 340/667] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:24.198647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4572d-01a3-45f1-9036-26af65d9a6ff None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:24.223901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032cc795-64dd-472d-a059-e4074e252513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:24.225646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cc795-64dd-472d-a059-e4074e252513 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:24.226214 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cc795-64dd-472d-a059-e4074e252513 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:24.226603 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cc795-64dd-472d-a059-e4074e252513 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:24.242147 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032cc795-64dd-472d-a059-e4074e252513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:24.243792 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032cc795-64dd-472d-a059-e4074e252513 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:24.263466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-032cc795-64dd-472d-a059-e4074e252513 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:24.263466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:24.263466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:24.263466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:24.263466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:24.263466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:24.263466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:24.263466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:24.263466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:24.263466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:24.263466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:24.263466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:24.263466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:24.263466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:24.263466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:24.269790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:24.269790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:24.269790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:24.269790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:24.269790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:24.269790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:24.269790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:24.269790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:24.269790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:24.269790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:24.269790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:24.269790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:24.269790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:24.269790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:24.269790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:24.269790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:24.269790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:24.271059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:24.271059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:24.271059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:24.271059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:24.271059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 328/668] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:24.271059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032cc795-64dd-472d-a059-e4074e252513 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:24.338059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4572d-01a3-45f1-9036-26af65d9a6ff None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:24.340083 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:24.391238 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032cc795-64dd-472d-a059-e4074e252513 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:24.398582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386253da-e15a-4f0e-980a-5ff02e1f932a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:24.399784 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386253da-e15a-4f0e-980a-5ff02e1f932a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:24.400311 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386253da-e15a-4f0e-980a-5ff02e1f932a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:24.400800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386253da-e15a-4f0e-980a-5ff02e1f932a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:24.438725 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97de4f-717e-4166-a535-e7280724b531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:24.439654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97de4f-717e-4166-a535-e7280724b531 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:24.439949 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97de4f-717e-4166-a535-e7280724b531 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:24.440230 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97de4f-717e-4166-a535-e7280724b531 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:24.441614 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97de4f-717e-4166-a535-e7280724b531 None admin] RBAC: Adding query filter params (role.id=9a48ab1bf32a45828e9e73599c980dac, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=ee2460e6a7e34a6881984e4f7ed294c1) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:24.442226 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97de4f-717e-4166-a535-e7280724b531 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:24.443963 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97de4f-717e-4166-a535-e7280724b531 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:24.451067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:24.456836 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 329/669] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:26:24 2022] GET /identity/v3/role_assignments?user.id=ee2460e6a7e34a6881984e4f7ed294c1&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=9a48ab1bf32a45828e9e73599c980dac => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:24.458206 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97de4f-717e-4166-a535-e7280724b531 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:24.511053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386253da-e15a-4f0e-980a-5ff02e1f932a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:24.513564 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386253da-e15a-4f0e-980a-5ff02e1f932a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:24.597417 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97de4f-717e-4166-a535-e7280724b531 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:24.639986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:24.663354 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 341/670] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:26:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:26:24.666314 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386253da-e15a-4f0e-980a-5ff02e1f932a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:24.670660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5417ce4-f2ff-4a24-a01e-921a56c91e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:24.671462 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5417ce4-f2ff-4a24-a01e-921a56c91e14 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:24.671746 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5417ce4-f2ff-4a24-a01e-921a56c91e14 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:24.672015 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5417ce4-f2ff-4a24-a01e-921a56c91e14 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:24.672899 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5417ce4-f2ff-4a24-a01e-921a56c91e14 None admin] RBAC: Adding query filter params (name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:24.673242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5417ce4-f2ff-4a24-a01e-921a56c91e14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:24.674809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5417ce4-f2ff-4a24-a01e-921a56c91e14 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:24.685766 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 330/671] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:26:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:24.686649 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5417ce4-f2ff-4a24-a01e-921a56c91e14 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:24.794931 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:24.835034 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5417ce4-f2ff-4a24-a01e-921a56c91e14 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:24.843801 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386253da-e15a-4f0e-980a-5ff02e1f932a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:24.889016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6e951-e79a-4e99-ac42-56bddf96b21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:24.889847 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6e951-e79a-4e99-ac42-56bddf96b21a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:24.890205 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6e951-e79a-4e99-ac42-56bddf96b21a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:24.890547 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6e951-e79a-4e99-ac42-56bddf96b21a None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:24.891328 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6e951-e79a-4e99-ac42-56bddf96b21a None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:24.891654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6e951-e79a-4e99-ac42-56bddf96b21a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:24.897258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6e951-e79a-4e99-ac42-56bddf96b21a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:24.903752 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c57889-d231-42d7-8fbc-f236899f2c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:24.906005 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c57889-d231-42d7-8fbc-f236899f2c51 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:24.906401 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c57889-d231-42d7-8fbc-f236899f2c51 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:24.906881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c57889-d231-42d7-8fbc-f236899f2c51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:24.910393 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 342/672] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:24.911479 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6e951-e79a-4e99-ac42-56bddf96b21a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:24.938083 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c57889-d231-42d7-8fbc-f236899f2c51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:24.941754 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c57889-d231-42d7-8fbc-f236899f2c51 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:24.967301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-40c57889-d231-42d7-8fbc-f236899f2c51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:24.967301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:24.967301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:24.967301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:24.967301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:24.967301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:24.967301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:24.967301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:24.967301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:24.967301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:24.967301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:24.967301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:24.967301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:24.967301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:24.967301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:24.969110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:24.969110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:24.969110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:24.969110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:24.969110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:24.969110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:24.969110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:24.969110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:24.969110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:24.969110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:24.969110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:24.969110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:24.969110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:24.969110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:24.969110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:24.969110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:24.969110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:24.971236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:24.971236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:24.971236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:24.971236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:24.973362 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 331/673] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:24.974384 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c57889-d231-42d7-8fbc-f236899f2c51 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:25.025150 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:25.093646 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6e951-e79a-4e99-ac42-56bddf96b21a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:25.120628 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c57889-d231-42d7-8fbc-f236899f2c51 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:25.146590 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:25.163279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093e4d0-83cc-4693-92a4-c6887432a304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:25.165124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093e4d0-83cc-4693-92a4-c6887432a304 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:25.165680 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093e4d0-83cc-4693-92a4-c6887432a304 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:25.166276 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093e4d0-83cc-4693-92a4-c6887432a304 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:25.169677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2cd64-a4c0-4cc6-be4f-7c504fe85f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:25.170707 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2cd64-a4c0-4cc6-be4f-7c504fe85f2b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:25.171473 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2cd64-a4c0-4cc6-be4f-7c504fe85f2b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:25.171906 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2cd64-a4c0-4cc6-be4f-7c504fe85f2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:25.180529 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2cd64-a4c0-4cc6-be4f-7c504fe85f2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:25.184048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2cd64-a4c0-4cc6-be4f-7c504fe85f2b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:25.191267 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093e4d0-83cc-4693-92a4-c6887432a304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:25.194193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093e4d0-83cc-4693-92a4-c6887432a304 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:25.204531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-67b2cd64-a4c0-4cc6-be4f-7c504fe85f2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:25.204531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:25.204531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:25.204531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:25.204531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:25.204531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:25.204531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:25.204531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:25.204531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:25.204531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:25.204531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:25.204531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:25.204531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:25.204531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:25.204531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:25.205544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:25.205544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:25.205544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:25.205544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:25.205544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:25.205544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:25.205544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:25.205544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:25.205544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:25.205544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:25.205544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:25.205544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:25.205544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:25.205544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:25.205544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:25.205544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:25.205544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:25.206453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:25.206453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:25.206453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:25.206453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:25.206453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:25.206453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:25.206453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:25.206453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:25.209242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 332/674] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:25.210217 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2cd64-a4c0-4cc6-be4f-7c504fe85f2b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:25.232636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-e093e4d0-83cc-4693-92a4-c6887432a304 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:25.232636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:25.232636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:25.232636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:25.232636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:25.232636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:25.232636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:25.232636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:25.232636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:25.232636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:25.232636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:25.232636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:25.232636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:25.232636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:25.232636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:25.234655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:25.234655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:25.234655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:25.234655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:25.234655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:25.234655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:25.234655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:25.234655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:25.234655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:25.234655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:25.234655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:25.234655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:25.234655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:25.234655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:25.234655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:25.234655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:25.234655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:25.235953 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:25.235953 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:25.235953 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:25.235953 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:25.235953 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 343/675] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:25.237804 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ad064-26f0-41db-9861-4ba5a066c246 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:25.238156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ad064-26f0-41db-9861-4ba5a066c246 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:25.238641 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ad064-26f0-41db-9861-4ba5a066c246 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:25.246060 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 344/676] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:25.246060 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ad064-26f0-41db-9861-4ba5a066c246 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:25.358817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:25.367049 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2cd64-a4c0-4cc6-be4f-7c504fe85f2b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:25.380035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ad064-26f0-41db-9861-4ba5a066c246 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:25.408560 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e267d-c140-46e5-8d16-15d9b0aeed66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:25.409402 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e267d-c140-46e5-8d16-15d9b0aeed66 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:25.409767 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e267d-c140-46e5-8d16-15d9b0aeed66 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:25.410123 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e267d-c140-46e5-8d16-15d9b0aeed66 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:25.410999 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e267d-c140-46e5-8d16-15d9b0aeed66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:25.411386 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e267d-c140-46e5-8d16-15d9b0aeed66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:25.412862 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e267d-c140-46e5-8d16-15d9b0aeed66 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:25.422642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c08910-80b8-400d-9f22-d0cee55f9626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:25.423523 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c08910-80b8-400d-9f22-d0cee55f9626 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:25.423870 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c08910-80b8-400d-9f22-d0cee55f9626 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:25.424247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c08910-80b8-400d-9f22-d0cee55f9626 None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:25.425148 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c08910-80b8-400d-9f22-d0cee55f9626 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:25.425527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c08910-80b8-400d-9f22-d0cee55f9626 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:25.427061 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c08910-80b8-400d-9f22-d0cee55f9626 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:25.427995 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 333/677] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:25.428866 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e267d-c140-46e5-8d16-15d9b0aeed66 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:25.453113 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 345/678] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:25.453113 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a1a1b-3479-4d4a-806e-6f7d0ff27e39 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:25.453113 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a1a1b-3479-4d4a-806e-6f7d0ff27e39 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:25.453754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a1a1b-3479-4d4a-806e-6f7d0ff27e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:25.481176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-299a1a1b-3479-4d4a-806e-6f7d0ff27e39 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:25.484009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-299a1a1b-3479-4d4a-806e-6f7d0ff27e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:25.538478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:25.581064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 346/679] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:25.581881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a1a1b-3479-4d4a-806e-6f7d0ff27e39 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:25.605993 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e267d-c140-46e5-8d16-15d9b0aeed66 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:25.654371 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9126e5a-3c29-4236-b603-6a8a871ba274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:25.655080 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9126e5a-3c29-4236-b603-6a8a871ba274 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:25.655381 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9126e5a-3c29-4236-b603-6a8a871ba274 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:25.655725 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9126e5a-3c29-4236-b603-6a8a871ba274 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:25.656544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9126e5a-3c29-4236-b603-6a8a871ba274 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:25.656905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9126e5a-3c29-4236-b603-6a8a871ba274 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:25.658324 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9126e5a-3c29-4236-b603-6a8a871ba274 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:25.671594 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 334/680] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:25.672452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9126e5a-3c29-4236-b603-6a8a871ba274 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:25.695349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:25.712041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a1a1b-3479-4d4a-806e-6f7d0ff27e39 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:25.754507 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dc11a-2e8f-4e9e-8449-591746a9f68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:25.755315 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dc11a-2e8f-4e9e-8449-591746a9f68f None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:25.755596 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dc11a-2e8f-4e9e-8449-591746a9f68f None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:25.755862 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dc11a-2e8f-4e9e-8449-591746a9f68f None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:25.763484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6dc11a-2e8f-4e9e-8449-591746a9f68f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:25.764959 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6dc11a-2e8f-4e9e-8449-591746a9f68f None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:25.779540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-4b6dc11a-2e8f-4e9e-8449-591746a9f68f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:25.779540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:25.779540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:25.779540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:25.779540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:25.779540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:25.779540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:25.779540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:25.779540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:25.779540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:25.779540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:25.779540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:25.779540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:25.779540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:25.779540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:25.781025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:25.781025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:25.781025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:25.781025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:25.781025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:25.781025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:25.781025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:25.781025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:25.781025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:25.781025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:25.781025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:25.781025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:25.781025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:25.781025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:25.781025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:25.781025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:25.781025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:25.782834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:25.782834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:25.782834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:25.782834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:25.782834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:25.782834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:25.782834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:25.782834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:25.782834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 347/681] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:25.782834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dc11a-2e8f-4e9e-8449-591746a9f68f None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:25.819545 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9126e5a-3c29-4236-b603-6a8a871ba274 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:25.829602 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:25.886479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c1265-ec91-4919-8414-44385aa8c964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:25.888134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c1265-ec91-4919-8414-44385aa8c964 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:25.888645 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c1265-ec91-4919-8414-44385aa8c964 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:25.889200 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c1265-ec91-4919-8414-44385aa8c964 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:25.890747 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c1265-ec91-4919-8414-44385aa8c964 None admin] RBAC: Adding query filter params (role.id=9a48ab1bf32a45828e9e73599c980dac, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=34db429cd31549f78daf5b1cbf6f0bf0) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:25.891200 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c1265-ec91-4919-8414-44385aa8c964 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:25.893504 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c1265-ec91-4919-8414-44385aa8c964 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:25.904981 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 335/682] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:26:25 2022] GET /identity/v3/role_assignments?user.id=34db429cd31549f78daf5b1cbf6f0bf0&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=9a48ab1bf32a45828e9e73599c980dac => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:25.906476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c1265-ec91-4919-8414-44385aa8c964 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:25.949807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dc11a-2e8f-4e9e-8449-591746a9f68f None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:26.019693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc49f1-2711-4308-b08a-2602b62dfffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:26.022849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc49f1-2711-4308-b08a-2602b62dfffa None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:26.024171 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc49f1-2711-4308-b08a-2602b62dfffa None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:26.025403 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc49f1-2711-4308-b08a-2602b62dfffa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:26.053808 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:26.061719 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc49f1-2711-4308-b08a-2602b62dfffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:26.074030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-f3cc49f1-2711-4308-b08a-2602b62dfffa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:26:26.074030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:26.074030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:26.074030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:26.074030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:26.074030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:26.074030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:26.074030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:26.074030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:26.074030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:26.074030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:26.074030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:26.074030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:26.074030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:26.074030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:26.074030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:26.075655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:26.075655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:26.075655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:26.075655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:26.075655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:26.075655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:26.075655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:26.075655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:26.075655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:26.075655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:26.075655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:26.075655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:26.075655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:26.075655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:26.075655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:26.075655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:26.076889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:26.076889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:26.076889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:26.076889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:26.076889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:26.076889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:26.076889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:26.076889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:26.076889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:26.076889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:26.076889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:26:26.076889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:26.076889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 348/683] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:26:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:26.076889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc49f1-2711-4308-b08a-2602b62dfffa None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:26.103052 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c1265-ec91-4919-8414-44385aa8c964 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:26.177482 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963aefb9-ece6-4236-8863-ef3f79033ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:26.178404 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963aefb9-ece6-4236-8863-ef3f79033ff9 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:26.178688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963aefb9-ece6-4236-8863-ef3f79033ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:26.178965 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963aefb9-ece6-4236-8863-ef3f79033ff9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:26.189599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963aefb9-ece6-4236-8863-ef3f79033ff9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:26.191654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963aefb9-ece6-4236-8863-ef3f79033ff9 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:26.210698 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:26.223812 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc49f1-2711-4308-b08a-2602b62dfffa None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:26.226568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-963aefb9-ece6-4236-8863-ef3f79033ff9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:26:26.226568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:26.226568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:26.226568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:26.226568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:26.226568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:26.226568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:26.226568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:26.226568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:26.226568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:26.226568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:26.226568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:26.226568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:26.226568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:26.226568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:26.226568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:26.228338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:26.228338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:26.228338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:26.228338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:26.228338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:26.228338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:26.228338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:26.228338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:26.228338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:26.228338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:26.228338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:26.228338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:26.228338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:26.228338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:26.228338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:26.228338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:26.228338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:26.228338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:26.230006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:26.230006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:26.230006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:26.230006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:26.230006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:26.230006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:26.230006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:26.230006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:26.230006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:26:26.230006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:26.230868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 336/684] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:26:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:26.239046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963aefb9-ece6-4236-8863-ef3f79033ff9 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:26.277653 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5d177-9a14-4565-b508-facfb30dbff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:26.278889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5d177-9a14-4565-b508-facfb30dbff5 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:26.279172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5d177-9a14-4565-b508-facfb30dbff5 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:26.279477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5d177-9a14-4565-b508-facfb30dbff5 None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:26.280303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5d177-9a14-4565-b508-facfb30dbff5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:26.280612 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5d177-9a14-4565-b508-facfb30dbff5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:26.282188 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5d177-9a14-4565-b508-facfb30dbff5 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:26.304031 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 349/685] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:26.304031 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5d177-9a14-4565-b508-facfb30dbff5 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:26.430739 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5d177-9a14-4565-b508-facfb30dbff5 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:26.444265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963aefb9-ece6-4236-8863-ef3f79033ff9 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:26.487282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d13a2-f252-4f64-af6a-0b7fd88a30d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:26.489301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d13a2-f252-4f64-af6a-0b7fd88a30d0 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:26.490037 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d13a2-f252-4f64-af6a-0b7fd88a30d0 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:26.490744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d13a2-f252-4f64-af6a-0b7fd88a30d0 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:26.493433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d13a2-f252-4f64-af6a-0b7fd88a30d0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:26.494633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d13a2-f252-4f64-af6a-0b7fd88a30d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:26.499036 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d13a2-f252-4f64-af6a-0b7fd88a30d0 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:26.511372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebceeead-3679-4d8a-af59-8e1895b73276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:26.512219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebceeead-3679-4d8a-af59-8e1895b73276 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:26.513228 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebceeead-3679-4d8a-af59-8e1895b73276 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:26.513614 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebceeead-3679-4d8a-af59-8e1895b73276 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:26.514921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebceeead-3679-4d8a-af59-8e1895b73276 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:26.515360 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebceeead-3679-4d8a-af59-8e1895b73276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:26.517254 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebceeead-3679-4d8a-af59-8e1895b73276 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:26.524585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:26.535227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 337/687] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:26:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:26.535426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 350/687] 10.222.0.71 () {56 vars in 1135 bytes} [Thu Jul 21 20:26:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:26.536575 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebceeead-3679-4d8a-af59-8e1895b73276 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:26.536737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d13a2-f252-4f64-af6a-0b7fd88a30d0 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:26.551302 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:26.665700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d13a2-f252-4f64-af6a-0b7fd88a30d0 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:26.706962 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cec0f4-fff9-4621-8cc8-faac6cf78f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:26.707873 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cec0f4-fff9-4621-8cc8-faac6cf78f47 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:26.708338 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cec0f4-fff9-4621-8cc8-faac6cf78f47 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:26.708765 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cec0f4-fff9-4621-8cc8-faac6cf78f47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:26.709915 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cec0f4-fff9-4621-8cc8-faac6cf78f47 None admin] RBAC: Adding query filter params (role.id=9a48ab1bf32a45828e9e73599c980dac, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=03eee8ef92314296b7d5c14c52f0d080) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:26.710415 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cec0f4-fff9-4621-8cc8-faac6cf78f47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:26.712081 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cec0f4-fff9-4621-8cc8-faac6cf78f47 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:26.725032 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebceeead-3679-4d8a-af59-8e1895b73276 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:26.725686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 351/688] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:26:26 2022] GET /identity/v3/role_assignments?user.id=03eee8ef92314296b7d5c14c52f0d080&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=9a48ab1bf32a45828e9e73599c980dac => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:26.726814 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cec0f4-fff9-4621-8cc8-faac6cf78f47 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:26.768595 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e506b6a-ff70-4e54-91eb-69e4b097a333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:26.769429 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e506b6a-ff70-4e54-91eb-69e4b097a333 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:26.769719 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e506b6a-ff70-4e54-91eb-69e4b097a333 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:26.770001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e506b6a-ff70-4e54-91eb-69e4b097a333 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:26.854246 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:26.857625 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e506b6a-ff70-4e54-91eb-69e4b097a333 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:26.860379 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e506b6a-ff70-4e54-91eb-69e4b097a333 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:26.882277 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:26.940592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cec0f4-fff9-4621-8cc8-faac6cf78f47 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:26.994252 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d586311-0c1a-45d6-8622-9e63151dafd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:26.994252 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d586311-0c1a-45d6-8622-9e63151dafd8 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:26.994535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d586311-0c1a-45d6-8622-9e63151dafd8 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:26.994772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d586311-0c1a-45d6-8622-9e63151dafd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:27.022118 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d586311-0c1a-45d6-8622-9e63151dafd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:27.023610 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d586311-0c1a-45d6-8622-9e63151dafd8 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:27.037511 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 338/689] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:26:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:26:27.039113 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261eb13-64ed-484c-ab65-0677d939ded6 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:27.039431 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261eb13-64ed-484c-ab65-0677d939ded6 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:27.039719 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261eb13-64ed-484c-ab65-0677d939ded6 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:27.041442 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 339/690] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:27.045813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-2d586311-0c1a-45d6-8622-9e63151dafd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:27.045813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:27.045813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:27.045813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:27.045813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:27.045813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:27.045813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:27.045813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:27.045813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:27.045813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:27.045813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:27.045813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:27.045813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:27.045813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:27.045813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:27.047120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:27.047120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:27.047120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:27.047120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:27.047120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:27.047120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:27.047120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:27.047120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:27.047120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:27.047120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:27.047120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:27.047120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:27.047120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:27.047120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:27.047120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:27.047120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:27.047120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:27.048511 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:27.048511 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:27.048511 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:27.048511 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:27.048511 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 352/691] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:27.053507 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261eb13-64ed-484c-ab65-0677d939ded6 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:27.054506 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d586311-0c1a-45d6-8622-9e63151dafd8 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:27.201674 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d586311-0c1a-45d6-8622-9e63151dafd8 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:27.209747 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:27.212553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261eb13-64ed-484c-ab65-0677d939ded6 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:27.230798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:27.244576 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa1a61-438a-41e7-acfc-a84184e59634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:27.245231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa1a61-438a-41e7-acfc-a84184e59634 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:27.245519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa1a61-438a-41e7-acfc-a84184e59634 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:27.245816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa1a61-438a-41e7-acfc-a84184e59634 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:27.246661 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa1a61-438a-41e7-acfc-a84184e59634 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:27.247062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa1a61-438a-41e7-acfc-a84184e59634 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:27.248495 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa1a61-438a-41e7-acfc-a84184e59634 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:27.259396 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7b547-6e12-42fe-9a68-1546522586f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:27.260494 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7b547-6e12-42fe-9a68-1546522586f7 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:27.260961 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7b547-6e12-42fe-9a68-1546522586f7 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:27.261444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7b547-6e12-42fe-9a68-1546522586f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:27.263374 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 353/692] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:27.264917 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be979e0b-dde8-46ab-9061-ac61265d3506 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:27.265238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be979e0b-dde8-46ab-9061-ac61265d3506 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:27.265518 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be979e0b-dde8-46ab-9061-ac61265d3506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:27.274968 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7b547-6e12-42fe-9a68-1546522586f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:27.276714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7b547-6e12-42fe-9a68-1546522586f7 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:27.286302 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-be979e0b-dde8-46ab-9061-ac61265d3506 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:27.287700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-be979e0b-dde8-46ab-9061-ac61265d3506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:27.297714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-69b7b547-6e12-42fe-9a68-1546522586f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:27.297714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:27.297714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:27.297714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:27.297714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:27.297714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:27.297714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:27.297714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:27.297714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:27.297714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:27.297714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:27.297714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:27.297714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:27.297714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:27.297714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:27.298881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:27.298881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:27.298881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:27.298881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:27.298881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:27.298881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:27.298881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:27.298881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:27.298881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:27.298881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:27.298881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:27.298881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:27.298881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:27.298881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:27.298881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:27.298881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:27.298881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:27.300004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:27.300004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:27.300004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:27.300004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:27.306632 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 340/693] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:27.307838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7b547-6e12-42fe-9a68-1546522586f7 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:27.377302 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 354/694] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:27.377302 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be979e0b-dde8-46ab-9061-ac61265d3506 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:27.475792 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7b547-6e12-42fe-9a68-1546522586f7 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:27.517257 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be979e0b-dde8-46ab-9061-ac61265d3506 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:27.529061 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f309a7c-dbe2-4bf3-a240-0cc809e0101e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:27.530153 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f309a7c-dbe2-4bf3-a240-0cc809e0101e None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:27.530559 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f309a7c-dbe2-4bf3-a240-0cc809e0101e None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:27.531711 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f309a7c-dbe2-4bf3-a240-0cc809e0101e None admin] PATH_INFO: `/v3/users/nova` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:27.538806 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:27.547746 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f309a7c-dbe2-4bf3-a240-0cc809e0101e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:27.549993 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f309a7c-dbe2-4bf3-a240-0cc809e0101e None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:27.573991 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0a0f0-92a2-4ed1-a7c3-b3198b0d3571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:27.574592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-7f309a7c-dbe2-4bf3-a240-0cc809e0101e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:26:27.574592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:27.574592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:27.574592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:27.574592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:27.574592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:27.574592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:27.574592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:27.574592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:27.574592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:27.574592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:27.574592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:27.574592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:27.574592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:27.574592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:27.574592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:27.575657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:27.575657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:27.575657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:27.575657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:27.575657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:27.575657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:27.575657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:27.575657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:27.575657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:27.575657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:27.575657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:27.575657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:27.575657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:27.575657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:27.575657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:27.575657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:27.575657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:27.575657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:27.576613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:27.576613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:27.576613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:27.576613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:27.576613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:27.576613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:27.576613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:27.576613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:27.576613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:26:27.576613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:27.576613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 341/695] 10.222.0.71 () {56 vars in 1093 bytes} [Thu Jul 21 20:26:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:27.578005 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f309a7c-dbe2-4bf3-a240-0cc809e0101e None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:27.578690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0a0f0-92a2-4ed1-a7c3-b3198b0d3571 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:27.579224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0a0f0-92a2-4ed1-a7c3-b3198b0d3571 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:27.579715 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0a0f0-92a2-4ed1-a7c3-b3198b0d3571 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:27.581181 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0a0f0-92a2-4ed1-a7c3-b3198b0d3571 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:27.581789 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0a0f0-92a2-4ed1-a7c3-b3198b0d3571 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:27.584786 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0a0f0-92a2-4ed1-a7c3-b3198b0d3571 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:27.597576 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:27.610974 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 355/696] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:27.610974 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0a0f0-92a2-4ed1-a7c3-b3198b0d3571 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:27.713358 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f309a7c-dbe2-4bf3-a240-0cc809e0101e None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:27.738385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0a0f0-92a2-4ed1-a7c3-b3198b0d3571 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:27.757708 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e6763-8461-41eb-8f60-2f6f6bef675f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:27.758911 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e6763-8461-41eb-8f60-2f6f6bef675f None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:27.759430 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e6763-8461-41eb-8f60-2f6f6bef675f None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:27.759948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e6763-8461-41eb-8f60-2f6f6bef675f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:27.772597 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e6763-8461-41eb-8f60-2f6f6bef675f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:27.774488 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e6763-8461-41eb-8f60-2f6f6bef675f None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:27.779725 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf9912-0735-48f4-89a6-8ce2e4801819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:27.780427 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf9912-0735-48f4-89a6-8ce2e4801819 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:27.780713 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf9912-0735-48f4-89a6-8ce2e4801819 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:27.781012 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf9912-0735-48f4-89a6-8ce2e4801819 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:27.782085 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf9912-0735-48f4-89a6-8ce2e4801819 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:27.782407 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf9912-0735-48f4-89a6-8ce2e4801819 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:27.783916 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf9912-0735-48f4-89a6-8ce2e4801819 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:27.798447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-e52e6763-8461-41eb-8f60-2f6f6bef675f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:27.798447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:27.798447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:27.798447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:27.798447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:27.798447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:27.798447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:27.798447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:27.798447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:27.798447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:27.798447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:27.798447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:27.798447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:27.798447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:27.798447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:27.799452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:27.799452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:27.799452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:27.799452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:27.799452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:27.799452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:27.799452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:27.799452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:27.799452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:27.799452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:27.799452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:27.799452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:27.799452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:27.799452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:27.799452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:27.799452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:27.799452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:27.800367 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:27.800367 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:27.800367 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:27.800367 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:27.807098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 356/697] 10.222.0.71 () {56 vars in 1133 bytes} [Thu Jul 21 20:26:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:27.807098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf9912-0735-48f4-89a6-8ce2e4801819 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:27.807876 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 342/698] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:27.809112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e6763-8461-41eb-8f60-2f6f6bef675f None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:27.842834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:27.886234 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:27.927001 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf9912-0735-48f4-89a6-8ce2e4801819 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:27.934205 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e6763-8461-41eb-8f60-2f6f6bef675f None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:27.968650 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72051d84-e8dd-4b2b-b3ea-db25d5857104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:27.969803 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72051d84-e8dd-4b2b-b3ea-db25d5857104 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:27.970241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72051d84-e8dd-4b2b-b3ea-db25d5857104 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:27.970711 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72051d84-e8dd-4b2b-b3ea-db25d5857104 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:27.975231 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5474807-2a0d-43bf-af3d-e271205edf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:27.976034 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5474807-2a0d-43bf-af3d-e271205edf6b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:27.976394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5474807-2a0d-43bf-af3d-e271205edf6b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:27.976748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5474807-2a0d-43bf-af3d-e271205edf6b None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:27.977708 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5474807-2a0d-43bf-af3d-e271205edf6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:27.978093 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5474807-2a0d-43bf-af3d-e271205edf6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:27.979692 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5474807-2a0d-43bf-af3d-e271205edf6b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:27.981840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72051d84-e8dd-4b2b-b3ea-db25d5857104 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:27.983810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72051d84-e8dd-4b2b-b3ea-db25d5857104 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:28.000151 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 343/699] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:28.000767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-72051d84-e8dd-4b2b-b3ea-db25d5857104 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:28.000767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:28.000767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:28.000767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:28.000767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:28.000767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:28.000767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:28.000767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:28.000767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:28.000767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:28.000767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:28.000767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:28.000767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:28.000767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:28.000767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:28.001721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:28.001721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:28.001721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:28.001721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:28.001721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:28.001721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:28.001721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:28.001721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:28.001721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:28.001721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:28.001721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:28.001721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:28.001721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:28.001721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:28.001721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:28.001721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:28.001721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:28.002629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:28.002629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:28.002629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:28.002629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:28.002629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:28.002629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:28.002629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:28.002629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:28.003060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5474807-2a0d-43bf-af3d-e271205edf6b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:28.005388 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 357/700] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:28.006558 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72051d84-e8dd-4b2b-b3ea-db25d5857104 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:28.121921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:28.124114 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5474807-2a0d-43bf-af3d-e271205edf6b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:28.126153 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72051d84-e8dd-4b2b-b3ea-db25d5857104 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:28.152033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:28.166217 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264500ad-c155-4cda-a424-3703cbf4049d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:28.166909 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264500ad-c155-4cda-a424-3703cbf4049d None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:28.167262 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264500ad-c155-4cda-a424-3703cbf4049d None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:28.167546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264500ad-c155-4cda-a424-3703cbf4049d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:28.169226 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50646bb2-694c-4208-8eb9-81cb23bbabea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:28.170225 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50646bb2-694c-4208-8eb9-81cb23bbabea None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:28.170724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50646bb2-694c-4208-8eb9-81cb23bbabea None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:28.171207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50646bb2-694c-4208-8eb9-81cb23bbabea None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:28.172140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50646bb2-694c-4208-8eb9-81cb23bbabea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:28.173899 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50646bb2-694c-4208-8eb9-81cb23bbabea None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:28.205585 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 358/701] 10.222.0.71 () {60 vars in 1132 bytes} [Thu Jul 21 20:26:28 2022] POST /identity/v3/users => generated 254 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:26:28.207043 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50646bb2-694c-4208-8eb9-81cb23bbabea None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:28.259722 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264500ad-c155-4cda-a424-3703cbf4049d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:28.261309 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264500ad-c155-4cda-a424-3703cbf4049d None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:28.380631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50646bb2-694c-4208-8eb9-81cb23bbabea None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:28.392592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 344/702] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:26:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:26:28.394208 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca186f9-a0e9-4282-ae39-21b7aa0ded7d None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:28.394502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca186f9-a0e9-4282-ae39-21b7aa0ded7d None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:28.394782 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca186f9-a0e9-4282-ae39-21b7aa0ded7d None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:28.396622 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 345/703] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:28.403199 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a72e2a-d1bc-4514-ac49-88397c6cba8f None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:28.403534 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a72e2a-d1bc-4514-ac49-88397c6cba8f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:28.403762 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a72e2a-d1bc-4514-ac49-88397c6cba8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:28.423993 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-e9a72e2a-d1bc-4514-ac49-88397c6cba8f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:28.425319 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-e9a72e2a-d1bc-4514-ac49-88397c6cba8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:28.456154 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:28.465312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483de2a1-1e11-4950-9bef-4b29896845be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:28.467146 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483de2a1-1e11-4950-9bef-4b29896845be None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:28.467627 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483de2a1-1e11-4950-9bef-4b29896845be None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:28.468106 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483de2a1-1e11-4950-9bef-4b29896845be None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:28.469705 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483de2a1-1e11-4950-9bef-4b29896845be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:28.470270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483de2a1-1e11-4950-9bef-4b29896845be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:28.473646 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483de2a1-1e11-4950-9bef-4b29896845be None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:28.498557 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 359/704] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:28.500078 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483de2a1-1e11-4950-9bef-4b29896845be None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:28.523394 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:28.548439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 346/705] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:26:28.549388 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a72e2a-d1bc-4514-ac49-88397c6cba8f None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:28.680608 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483de2a1-1e11-4950-9bef-4b29896845be None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:28.714047 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a72e2a-d1bc-4514-ac49-88397c6cba8f None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:28.753697 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf39c4-67ab-4d37-b8b4-8e169685cbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:28.755325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf39c4-67ab-4d37-b8b4-8e169685cbfc None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:28.755813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf39c4-67ab-4d37-b8b4-8e169685cbfc None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:28.756288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf39c4-67ab-4d37-b8b4-8e169685cbfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:28.760927 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac122332-2a06-469d-939a-e877925eca45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:28.762149 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac122332-2a06-469d-939a-e877925eca45 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:28.762972 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac122332-2a06-469d-939a-e877925eca45 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:28.763253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac122332-2a06-469d-939a-e877925eca45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:28.764132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac122332-2a06-469d-939a-e877925eca45 None admin] RBAC: Adding query filter params (role.id=9a48ab1bf32a45828e9e73599c980dac, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=3babb078203647e2b3ef3b2787b84e3b) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:28.764449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac122332-2a06-469d-939a-e877925eca45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:28.765989 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac122332-2a06-469d-939a-e877925eca45 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:28.774904 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 347/706] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:26:28 2022] GET /identity/v3/role_assignments?user.id=3babb078203647e2b3ef3b2787b84e3b&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=9a48ab1bf32a45828e9e73599c980dac => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:28.775103 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf39c4-67ab-4d37-b8b4-8e169685cbfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:28.775856 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac122332-2a06-469d-939a-e877925eca45 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:28.778039 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf39c4-67ab-4d37-b8b4-8e169685cbfc None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:28.803608 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:28.820196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-91bf39c4-67ab-4d37-b8b4-8e169685cbfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:28.820196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:28.820196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:28.820196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:28.820196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:28.820196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:28.820196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:28.820196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:28.820196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:28.820196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:28.820196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:28.820196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:28.820196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:28.820196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:28.820196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:28.821433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:28.821433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:28.821433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:28.821433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:28.821433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:28.821433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:28.821433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:28.821433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:28.821433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:28.821433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:28.821433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:28.821433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:28.821433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:28.821433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:28.821433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:28.821433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:28.821433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:28.822395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:28.822395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:28.822395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:28.822395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:28.823041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 360/707] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:28.834692 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf39c4-67ab-4d37-b8b4-8e169685cbfc None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:28.920310 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac122332-2a06-469d-939a-e877925eca45 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:28.966377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282df37-af60-47ff-8d7b-c140bc75f721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:28.967135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282df37-af60-47ff-8d7b-c140bc75f721 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:28.967474 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282df37-af60-47ff-8d7b-c140bc75f721 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:28.968086 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282df37-af60-47ff-8d7b-c140bc75f721 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:28.968847 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282df37-af60-47ff-8d7b-c140bc75f721 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:28.970545 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282df37-af60-47ff-8d7b-c140bc75f721 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:28.993116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-b282df37-af60-47ff-8d7b-c140bc75f721 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 20:26:28.993116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:28.993116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:26:28.993116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:26:28.993116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:26:28.993116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:26:28.993116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:26:28.993116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:28.993116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 20:26:28.993116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:28.993116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:28.993116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:28.993116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:28.993116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:28.993116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:28.993116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:28.994224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:28.994224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:28.994224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:28.994224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:28.994224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:28.994224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:28.994224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:28.994224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:26:28.994224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:26:28.994224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:26:28.994224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:26:28.994224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:28.994224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:28.994224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:28.994224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:28.994224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:28.995132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:28.995132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:28.995132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:28.995132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:28.995132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:28.995132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:28.995132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:28.995132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:28.995132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:28.995132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:28.995132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:28.995132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:26:28.995132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:28.995132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 20:26:28.995132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:28.995132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 348/708] 10.222.0.71 () {56 vars in 1129 bytes} [Thu Jul 21 20:26:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:28.996016 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282df37-af60-47ff-8d7b-c140bc75f721 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:29.037526 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:29.097902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf39c4-67ab-4d37-b8b4-8e169685cbfc None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:29.141445 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:29.144414 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a68fd-51c3-416f-b056-1ecb82c55237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:29.145467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a68fd-51c3-416f-b056-1ecb82c55237 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:29.146399 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a68fd-51c3-416f-b056-1ecb82c55237 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:29.146901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a68fd-51c3-416f-b056-1ecb82c55237 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:29.147866 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a68fd-51c3-416f-b056-1ecb82c55237 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:29.148589 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a68fd-51c3-416f-b056-1ecb82c55237 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:29.150708 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a68fd-51c3-416f-b056-1ecb82c55237 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:29.169299 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 361/709] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:29.174411 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a68fd-51c3-416f-b056-1ecb82c55237 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:29.182269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282df37-af60-47ff-8d7b-c140bc75f721 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:29.229545 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb111c-37fd-45ec-94f4-f8d7649cc389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:29.230615 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb111c-37fd-45ec-94f4-f8d7649cc389 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:29.231162 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb111c-37fd-45ec-94f4-f8d7649cc389 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:29.231867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb111c-37fd-45ec-94f4-f8d7649cc389 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:29.232856 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb111c-37fd-45ec-94f4-f8d7649cc389 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:29.234454 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb111c-37fd-45ec-94f4-f8d7649cc389 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:29.236520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb111c-37fd-45ec-94f4-f8d7649cc389 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:29.248516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 349/710] 10.222.0.71 () {56 vars in 1124 bytes} [Thu Jul 21 20:26:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:29.251833 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb111c-37fd-45ec-94f4-f8d7649cc389 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:29.384002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a68fd-51c3-416f-b056-1ecb82c55237 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:29.390810 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb111c-37fd-45ec-94f4-f8d7649cc389 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:29.427116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:29.435138 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8a2a4-573c-4b7b-bfa1-9336bf992d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:29.436402 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8a2a4-573c-4b7b-bfa1-9336bf992d74 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:29.437476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8a2a4-573c-4b7b-bfa1-9336bf992d74 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:29.437995 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8a2a4-573c-4b7b-bfa1-9336bf992d74 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:29.439240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8a2a4-573c-4b7b-bfa1-9336bf992d74 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:29.439767 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8a2a4-573c-4b7b-bfa1-9336bf992d74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:29.441830 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8a2a4-573c-4b7b-bfa1-9336bf992d74 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:29.461846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a0cfb-2d37-49b6-9d23-5201af837a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:29.462992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a0cfb-2d37-49b6-9d23-5201af837a09 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:29.465186 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 350/711] 10.222.0.71 () {56 vars in 1124 bytes} [Thu Jul 21 20:26:29 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:29.470867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a0cfb-2d37-49b6-9d23-5201af837a09 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:29.471587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a0cfb-2d37-49b6-9d23-5201af837a09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:29.493626 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e8ff9-8698-41c8-9fa9-259bc629669e None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:29.493831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e8ff9-8698-41c8-9fa9-259bc629669e None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:29.494102 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e8ff9-8698-41c8-9fa9-259bc629669e None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:29.496032 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 351/712] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:26:29.502058 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690eaaa4-0644-4822-935b-fa0f7b2e8ef8 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:29.502387 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690eaaa4-0644-4822-935b-fa0f7b2e8ef8 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:29.502631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690eaaa4-0644-4822-935b-fa0f7b2e8ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:29.506481 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a0cfb-2d37-49b6-9d23-5201af837a09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:29.511902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a0cfb-2d37-49b6-9d23-5201af837a09 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:29.533824 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-690eaaa4-0644-4822-935b-fa0f7b2e8ef8 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:29.541753 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-690eaaa4-0644-4822-935b-fa0f7b2e8ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:29.548876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-8a0a0cfb-2d37-49b6-9d23-5201af837a09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:29.548876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:29.548876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:29.548876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:29.548876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:29.548876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:29.548876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:29.548876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:29.548876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:29.548876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:29.548876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:29.548876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:29.548876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:29.548876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:29.548876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:29.550035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:29.550035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:29.550035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:29.550035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:29.550035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:29.550035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:29.550035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:29.550035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:29.550035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:29.550035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:29.550035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:29.550035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:29.550035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:29.550035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:29.550035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:29.550035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:29.550035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:29.551224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:29.551224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:29.551224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:29.551224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:29.551224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:29.551224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:29.551224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:29.551224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:29.551660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:29.551745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 362/713] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:29.557855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a0cfb-2d37-49b6-9d23-5201af837a09 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:29.718307 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 352/714] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:26:29.741227 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a0cfb-2d37-49b6-9d23-5201af837a09 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:29.743585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690eaaa4-0644-4822-935b-fa0f7b2e8ef8 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:29.798127 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a528851-e09a-4c67-ad33-b0a69777dc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:29.798747 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a528851-e09a-4c67-ad33-b0a69777dc75 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:29.799032 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a528851-e09a-4c67-ad33-b0a69777dc75 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:29.799314 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a528851-e09a-4c67-ad33-b0a69777dc75 None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:29.800161 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a528851-e09a-4c67-ad33-b0a69777dc75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:29.800475 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a528851-e09a-4c67-ad33-b0a69777dc75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:29.802837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a528851-e09a-4c67-ad33-b0a69777dc75 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:29.832126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 363/715] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:29.850830 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a528851-e09a-4c67-ad33-b0a69777dc75 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:29.904369 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:29.984465 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690eaaa4-0644-4822-935b-fa0f7b2e8ef8 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:29.988621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a528851-e09a-4c67-ad33-b0a69777dc75 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:30.031710 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7048a8-c8db-4071-b2fb-e50ee7ba6e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:30.033032 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7048a8-c8db-4071-b2fb-e50ee7ba6e9e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:30.033417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7048a8-c8db-4071-b2fb-e50ee7ba6e9e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:30.033697 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7048a8-c8db-4071-b2fb-e50ee7ba6e9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:30.034596 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7048a8-c8db-4071-b2fb-e50ee7ba6e9e None admin] RBAC: Adding query filter params (role.id=19f5e72c0be44a1197c7288de82e79e8, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=6fbc5240f5214aaea4418052b94739e2) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:30.034919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7048a8-c8db-4071-b2fb-e50ee7ba6e9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:30.036393 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7048a8-c8db-4071-b2fb-e50ee7ba6e9e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:30.049687 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 364/716] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:26:29 2022] GET /identity/v3/role_assignments?user.id=6fbc5240f5214aaea4418052b94739e2&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:30.103701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974260d2-d625-4c90-a0c1-9da848a5f223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:30.104539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974260d2-d625-4c90-a0c1-9da848a5f223 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:30.104875 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974260d2-d625-4c90-a0c1-9da848a5f223 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:30.105242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974260d2-d625-4c90-a0c1-9da848a5f223 None admin] PATH_INFO: `/v3/services/network` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:30.105953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974260d2-d625-4c90-a0c1-9da848a5f223 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:30.107995 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974260d2-d625-4c90-a0c1-9da848a5f223 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:30.121236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-974260d2-d625-4c90-a0c1-9da848a5f223 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 20:26:30.121236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:30.121236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:26:30.121236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:26:30.121236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:26:30.121236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:26:30.121236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:26:30.121236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:30.121236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 20:26:30.121236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:30.121236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:30.121236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:30.121236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:30.121236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:30.121236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:30.121236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:30.121236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:30.123777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:30.123777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:30.123777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:30.123777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:30.123777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:30.123777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:30.123777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:26:30.123777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:26:30.123777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:26:30.123777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:26:30.123777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:30.123777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:30.123777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:30.123777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:30.123777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:30.123777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:30.125910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:30.125910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:30.125910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:30.125910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:30.125910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:30.125910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:30.125910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:30.125910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:30.125910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:30.125910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:30.125910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:26:30.125910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:30.125910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 20:26:30.125910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:30.125910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 353/717] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:29 2022] GET /identity/v3/services/network => generated 88 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:26:30.133162 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7048a8-c8db-4071-b2fb-e50ee7ba6e9e None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:30.276534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7048a8-c8db-4071-b2fb-e50ee7ba6e9e None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:30.282518 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:30.335370 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd5d54-5bfc-4016-ac9d-c81a21bc72c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:30.336031 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd5d54-5bfc-4016-ac9d-c81a21bc72c9 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:30.336318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd5d54-5bfc-4016-ac9d-c81a21bc72c9 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:30.336595 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd5d54-5bfc-4016-ac9d-c81a21bc72c9 None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:30.337414 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd5d54-5bfc-4016-ac9d-c81a21bc72c9 None admin] RBAC: Adding query filter params (name=network) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:30.337730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd5d54-5bfc-4016-ac9d-c81a21bc72c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:30.339227 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd5d54-5bfc-4016-ac9d-c81a21bc72c9 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:30.350209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 365/718] 10.222.0.71 () {56 vars in 1112 bytes} [Thu Jul 21 20:26:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:30.355692 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974260d2-d625-4c90-a0c1-9da848a5f223 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:30.360375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:30.484992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974260d2-d625-4c90-a0c1-9da848a5f223 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:30.523654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eacf8c-ed35-4344-b9e3-0ac5d173b45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:30.524388 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eacf8c-ed35-4344-b9e3-0ac5d173b45f None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:30.524678 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eacf8c-ed35-4344-b9e3-0ac5d173b45f None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:30.524994 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eacf8c-ed35-4344-b9e3-0ac5d173b45f None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:30.525767 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eacf8c-ed35-4344-b9e3-0ac5d173b45f None admin] RBAC: Adding query filter params (type=network) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:30.526077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eacf8c-ed35-4344-b9e3-0ac5d173b45f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:30.527495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eacf8c-ed35-4344-b9e3-0ac5d173b45f None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:30.534026 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 354/719] 10.222.0.71 () {56 vars in 1112 bytes} [Thu Jul 21 20:26:30 2022] GET /identity/v3/services?type=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:30.797837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164f80d-7e3e-4fc0-a787-9d30b4248628 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:30.798215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164f80d-7e3e-4fc0-a787-9d30b4248628 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:30.798407 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164f80d-7e3e-4fc0-a787-9d30b4248628 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:30.799800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 366/720] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:30.805740 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e66e5-bb86-4e1c-92e3-cdd6323a9acd None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:30.806041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e66e5-bb86-4e1c-92e3-cdd6323a9acd None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:30.806292 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e66e5-bb86-4e1c-92e3-cdd6323a9acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:30.844291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-e42e66e5-bb86-4e1c-92e3-cdd6323a9acd None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:30.846928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-e42e66e5-bb86-4e1c-92e3-cdd6323a9acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:30.977739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:30.984578 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 355/721] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:31.022754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164f80d-7e3e-4fc0-a787-9d30b4248628 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:31.173969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164f80d-7e3e-4fc0-a787-9d30b4248628 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:31.224167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb18b53-77bc-462b-9f74-705255c83be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:31.225284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb18b53-77bc-462b-9f74-705255c83be9 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:31.225570 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb18b53-77bc-462b-9f74-705255c83be9 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:31.225839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb18b53-77bc-462b-9f74-705255c83be9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:31.245096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb18b53-77bc-462b-9f74-705255c83be9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:31.253772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-6eb18b53-77bc-462b-9f74-705255c83be9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:31.253772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:31.253772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:31.253772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:31.253772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:31.253772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:31.253772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:31.253772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:31.253772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:31.253772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:31.253772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:31.253772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:31.253772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:31.253772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:31.253772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:31.253772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:31.255611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:31.255611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:31.255611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:31.255611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:31.255611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:31.255611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:31.255611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:31.255611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:31.255611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:31.255611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:31.255611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:31.255611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:31.255611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:31.255611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:31.255611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:31.255611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:31.257994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:31.257994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:31.257994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:31.257994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:31.257994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:31.257994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:31.257994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:31.257994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:31.257994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:31.257994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:31.257994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:31.257994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:31.257994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 367/722] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:31.266438 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e66e5-bb86-4e1c-92e3-cdd6323a9acd None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:31.424632 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e66e5-bb86-4e1c-92e3-cdd6323a9acd None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:31.510971 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795dc56-5b77-4242-971f-142d36d016dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:31.511756 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795dc56-5b77-4242-971f-142d36d016dc None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:31.512106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795dc56-5b77-4242-971f-142d36d016dc None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:31.512480 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795dc56-5b77-4242-971f-142d36d016dc None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:31.513340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e795dc56-5b77-4242-971f-142d36d016dc None admin] RBAC: Adding query filter params (name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:31.513663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e795dc56-5b77-4242-971f-142d36d016dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:31.515111 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e795dc56-5b77-4242-971f-142d36d016dc None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:31.528418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 356/723] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:26:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:31.534194 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb18b53-77bc-462b-9f74-705255c83be9 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:31.577098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:31.682416 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb18b53-77bc-462b-9f74-705255c83be9 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:31.734624 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93419e-2f2b-4126-806f-6fc74fe0f86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:31.735444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93419e-2f2b-4126-806f-6fc74fe0f86e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:31.735730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93419e-2f2b-4126-806f-6fc74fe0f86e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:31.736009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93419e-2f2b-4126-806f-6fc74fe0f86e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:31.766023 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93419e-2f2b-4126-806f-6fc74fe0f86e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:31.766023 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93419e-2f2b-4126-806f-6fc74fe0f86e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:31.780982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-8a93419e-2f2b-4126-806f-6fc74fe0f86e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:31.780982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:31.780982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:31.780982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:31.780982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:31.780982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:31.780982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:31.780982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:31.780982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:31.780982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:31.780982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:31.780982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:31.780982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:31.780982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:31.780982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:31.782734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:31.782734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:31.782734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:31.782734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:31.782734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:31.782734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:31.782734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:31.782734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:31.782734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:31.782734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:31.782734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:31.782734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:31.782734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:31.782734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:31.782734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:31.782734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:31.782734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:31.784323 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:31.784323 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:31.784323 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:31.784323 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:31.784323 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 368/724] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:31.788360 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795dc56-5b77-4242-971f-142d36d016dc None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:31.882583 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:31.930734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795dc56-5b77-4242-971f-142d36d016dc None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:31.982118 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e368e-fe09-4cb0-960f-055660437d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:31.982721 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e368e-fe09-4cb0-960f-055660437d58 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:31.983005 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e368e-fe09-4cb0-960f-055660437d58 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:31.983289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e368e-fe09-4cb0-960f-055660437d58 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:31.984034 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e368e-fe09-4cb0-960f-055660437d58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:31.984344 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e368e-fe09-4cb0-960f-055660437d58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:31.985777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e368e-fe09-4cb0-960f-055660437d58 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:32.010852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 357/725] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:32.017983 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93419e-2f2b-4126-806f-6fc74fe0f86e None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:32.115146 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:32.148315 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93419e-2f2b-4126-806f-6fc74fe0f86e None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:32.189715 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf020d28-f92c-4f3e-a057-1e4688a022d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:32.190593 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf020d28-f92c-4f3e-a057-1e4688a022d3 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:32.190925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf020d28-f92c-4f3e-a057-1e4688a022d3 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:32.191205 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf020d28-f92c-4f3e-a057-1e4688a022d3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:32.200293 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf020d28-f92c-4f3e-a057-1e4688a022d3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:32.201879 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf020d28-f92c-4f3e-a057-1e4688a022d3 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:32.225207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-cf020d28-f92c-4f3e-a057-1e4688a022d3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:26:32.225207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:32.225207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:32.225207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:32.225207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:32.225207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:32.225207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:32.225207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:32.225207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:32.225207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:32.225207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:32.225207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:32.225207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:32.225207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:32.225207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:32.225207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:32.226215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:32.226215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:32.226215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:32.226215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:32.226215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:32.226215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:32.226215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:32.226215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:32.226215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:32.226215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:32.226215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:32.226215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:32.226215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:32.226215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:32.226215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:32.226215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:32.226215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:32.226215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:32.227693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:32.227693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:32.227693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:32.227693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:32.227693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:32.227693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:32.227693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:32.227693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:32.227693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:26:32.227693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:32.227693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 369/726] 10.222.0.71 () {56 vars in 1117 bytes} [Thu Jul 21 20:26:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:32.232599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e368e-fe09-4cb0-960f-055660437d58 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:32.311150 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d37b9-2b86-4246-8d73-a08e6b042714 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:32.311564 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d37b9-2b86-4246-8d73-a08e6b042714 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:32.311723 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d37b9-2b86-4246-8d73-a08e6b042714 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:32.313201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 370/727] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:26:32.322852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bd5f6-ed63-4b58-ba26-1e307a97d629 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:32.323790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bd5f6-ed63-4b58-ba26-1e307a97d629 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:32.324569 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bd5f6-ed63-4b58-ba26-1e307a97d629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:32.365736 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-7f6bd5f6-ed63-4b58-ba26-1e307a97d629 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:32.368263 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-7f6bd5f6-ed63-4b58-ba26-1e307a97d629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:32.404405 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e368e-fe09-4cb0-960f-055660437d58 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:32.428617 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:32.461096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83314414-3a96-499f-aafa-7234eb0de442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:32.461096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83314414-3a96-499f-aafa-7234eb0de442 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:32.461096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83314414-3a96-499f-aafa-7234eb0de442 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:32.461431 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83314414-3a96-499f-aafa-7234eb0de442 None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:32.462304 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83314414-3a96-499f-aafa-7234eb0de442 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:32.462627 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83314414-3a96-499f-aafa-7234eb0de442 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:32.464109 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83314414-3a96-499f-aafa-7234eb0de442 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:32.504623 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 358/728] 10.222.0.71 () {56 vars in 1149 bytes} [Thu Jul 21 20:26:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:32.512077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83314414-3a96-499f-aafa-7234eb0de442 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:32.518982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 371/729] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:32.520386 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b3eee-7dc8-462c-804c-27cca52fa928 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:32.520679 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b3eee-7dc8-462c-804c-27cca52fa928 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:32.520961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b3eee-7dc8-462c-804c-27cca52fa928 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:32.522703 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 372/730] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:32.528376 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb65ee1-6595-486e-b2a8-3017e040018d None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:32.528686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb65ee1-6595-486e-b2a8-3017e040018d None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:32.528987 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb65ee1-6595-486e-b2a8-3017e040018d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:32.548230 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-fcb65ee1-6595-486e-b2a8-3017e040018d None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:32.549498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-fcb65ee1-6595-486e-b2a8-3017e040018d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:32.625128 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:32.629285 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 373/731] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:26:32.631723 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb65ee1-6595-486e-b2a8-3017e040018d None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:32.682273 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83314414-3a96-499f-aafa-7234eb0de442 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:32.722232 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0e2cd-eeac-4c5e-9e91-63ef0ff282be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:32.723127 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0e2cd-eeac-4c5e-9e91-63ef0ff282be None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:32.723243 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0e2cd-eeac-4c5e-9e91-63ef0ff282be None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:32.723450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0e2cd-eeac-4c5e-9e91-63ef0ff282be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:32.750384 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb65ee1-6595-486e-b2a8-3017e040018d None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:32.778859 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:32.791828 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdb75d-2f4d-46c5-b82b-b3d3594dda8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:32.793268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdb75d-2f4d-46c5-b82b-b3d3594dda8d None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:32.794017 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdb75d-2f4d-46c5-b82b-b3d3594dda8d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:32.794523 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdb75d-2f4d-46c5-b82b-b3d3594dda8d None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:32.796108 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdb75d-2f4d-46c5-b82b-b3d3594dda8d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:32.797958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdb75d-2f4d-46c5-b82b-b3d3594dda8d None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:32.805938 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0e2cd-eeac-4c5e-9e91-63ef0ff282be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:32.807856 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0e2cd-eeac-4c5e-9e91-63ef0ff282be None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:32.820932 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 374/732] 10.222.0.71 () {60 vars in 1141 bytes} [Thu Jul 21 20:26:32 2022] POST /identity/v3/services => generated 251 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:26:32.822802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdb75d-2f4d-46c5-b82b-b3d3594dda8d None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:32.914151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:32.984271 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 359/733] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:26:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 20:26:32.986067 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0e2cd-eeac-4c5e-9e91-63ef0ff282be None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:33.029401 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdb75d-2f4d-46c5-b82b-b3d3594dda8d None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:33.080245 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c60817-ee76-426c-b486-370fe2b06d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:33.081502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c60817-ee76-426c-b486-370fe2b06d64 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:33.082019 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c60817-ee76-426c-b486-370fe2b06d64 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:33.082511 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c60817-ee76-426c-b486-370fe2b06d64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:33.095670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c60817-ee76-426c-b486-370fe2b06d64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:33.102641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:33.107297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-f1c60817-ee76-426c-b486-370fe2b06d64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:26:33.107297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:33.107297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:33.107297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:33.107297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:33.107297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:33.107297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:33.107297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:33.107297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:33.107297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:33.107297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:33.107297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:33.107297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:33.107297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:33.107297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:33.107297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:33.108668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:33.108668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:33.108668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:33.108668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:33.108668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:33.108668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:33.108668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:33.108668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:33.108668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:33.108668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:33.108668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:33.108668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:33.108668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:33.108668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:33.108668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:33.108668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:33.109563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:33.109563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:33.109563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:33.109563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:33.109563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:33.109563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:33.109563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:33.109563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:33.109563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:33.109563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:33.109563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:26:33.109563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:33.113185 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 375/734] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:26:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:33.122336 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c60817-ee76-426c-b486-370fe2b06d64 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:33.134656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0e2cd-eeac-4c5e-9e91-63ef0ff282be None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:33.191806 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d83a92-c37d-4e3a-8690-a720e7bcc2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:33.192685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d83a92-c37d-4e3a-8690-a720e7bcc2b6 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:33.192888 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d83a92-c37d-4e3a-8690-a720e7bcc2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:33.193774 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d83a92-c37d-4e3a-8690-a720e7bcc2b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:33.206923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d83a92-c37d-4e3a-8690-a720e7bcc2b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:33.207802 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d83a92-c37d-4e3a-8690-a720e7bcc2b6 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:33.232781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-78d83a92-c37d-4e3a-8690-a720e7bcc2b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:33.232781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:33.232781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:33.232781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:33.232781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:33.232781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:33.232781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:33.232781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:33.232781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:33.232781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:33.232781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:33.232781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:33.232781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:33.232781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:33.232781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:33.234815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:33.234815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:33.234815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:33.234815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:33.234815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:33.234815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:33.234815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:33.234815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:33.234815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:33.234815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:33.234815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:33.234815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:33.234815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:33.234815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:33.234815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:33.234815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:33.234815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:33.235787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:33.235787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:33.235787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:33.235787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:33.235787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 360/735] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:33.235787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ce717-48fb-47b1-be90-17900460f649 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:33.235787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ce717-48fb-47b1-be90-17900460f649 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:33.235787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ce717-48fb-47b1-be90-17900460f649 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:33.241182 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 361/736] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:33.242488 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eff516-86e8-4936-b0bd-d761314abe0f None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:33.242770 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eff516-86e8-4936-b0bd-d761314abe0f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:33.243041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eff516-86e8-4936-b0bd-d761314abe0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:33.261451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-97eff516-86e8-4936-b0bd-d761314abe0f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:33.262719 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-97eff516-86e8-4936-b0bd-d761314abe0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:33.266568 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c60817-ee76-426c-b486-370fe2b06d64 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:33.285560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:33.305824 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec06a77-0118-4be7-8177-0708fb2f5666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:33.306500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec06a77-0118-4be7-8177-0708fb2f5666 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:33.306783 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec06a77-0118-4be7-8177-0708fb2f5666 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:33.307063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec06a77-0118-4be7-8177-0708fb2f5666 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:33.307812 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec06a77-0118-4be7-8177-0708fb2f5666 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:33.308119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec06a77-0118-4be7-8177-0708fb2f5666 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:33.310140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec06a77-0118-4be7-8177-0708fb2f5666 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:33.319136 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 376/737] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:26:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:33.320023 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec06a77-0118-4be7-8177-0708fb2f5666 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:33.339017 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 362/738] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:33 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:33.340481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fa843-dbae-4713-b383-5d043efc4df7 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:33.340780 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fa843-dbae-4713-b383-5d043efc4df7 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:33.341033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fa843-dbae-4713-b383-5d043efc4df7 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:33.346683 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 363/739] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:33.346683 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fa843-dbae-4713-b383-5d043efc4df7 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:33.413131 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:33.429859 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec06a77-0118-4be7-8177-0708fb2f5666 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:33.466245 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fa843-dbae-4713-b383-5d043efc4df7 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:33.469210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6b9d3-46f8-40ae-b7c1-75313c80952e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:33.469879 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6b9d3-46f8-40ae-b7c1-75313c80952e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:33.470234 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6b9d3-46f8-40ae-b7c1-75313c80952e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:33.470607 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6b9d3-46f8-40ae-b7c1-75313c80952e None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:33.471248 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6b9d3-46f8-40ae-b7c1-75313c80952e None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:33.471610 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6b9d3-46f8-40ae-b7c1-75313c80952e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:33.475596 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6b9d3-46f8-40ae-b7c1-75313c80952e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:33.493880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 377/740] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:33.495281 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967ea4c-ff34-49f5-ad68-9f852be823ef None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:33.495575 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967ea4c-ff34-49f5-ad68-9f852be823ef None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:33.495857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967ea4c-ff34-49f5-ad68-9f852be823ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:33.505972 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd056e8-33be-4984-9a1c-4a182201506a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:33.506760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd056e8-33be-4984-9a1c-4a182201506a None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:33.507038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd056e8-33be-4984-9a1c-4a182201506a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:33.507309 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd056e8-33be-4984-9a1c-4a182201506a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:33.514530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-d967ea4c-ff34-49f5-ad68-9f852be823ef None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:33.515731 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-d967ea4c-ff34-49f5-ad68-9f852be823ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:33.518886 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd056e8-33be-4984-9a1c-4a182201506a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:33.520338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd056e8-33be-4984-9a1c-4a182201506a None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:33.538803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-9fd056e8-33be-4984-9a1c-4a182201506a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:33.538803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:33.538803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:33.538803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:33.538803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:33.538803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:33.538803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:33.538803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:33.538803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:33.538803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:33.538803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:33.538803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:33.538803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:33.538803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:33.538803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:33.540001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:33.540001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:33.540001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:33.540001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:33.540001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:33.540001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:33.540001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:33.540001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:33.540001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:33.540001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:33.540001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:33.540001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:33.540001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:33.540001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:33.540001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:33.540001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:33.540001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:33.541165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:33.541165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:33.541165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:33.541165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:33.545203 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 364/741] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:33.546092 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd056e8-33be-4984-9a1c-4a182201506a None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:33.551733 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:33.593873 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 378/742] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:33 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:26:33.594628 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967ea4c-ff34-49f5-ad68-9f852be823ef None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:33.686096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd056e8-33be-4984-9a1c-4a182201506a None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:33.701946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:33.706798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967ea4c-ff34-49f5-ad68-9f852be823ef None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:33.727964 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5059eb0-c6bd-4441-8ff3-01e289265b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:33.728756 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5059eb0-c6bd-4441-8ff3-01e289265b08 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:33.729031 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5059eb0-c6bd-4441-8ff3-01e289265b08 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:33.729370 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5059eb0-c6bd-4441-8ff3-01e289265b08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:33.741325 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5059eb0-c6bd-4441-8ff3-01e289265b08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:33.745547 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc9b69-da89-465e-8af9-67dccb9a3a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:33.746590 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc9b69-da89-465e-8af9-67dccb9a3a77 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:33.747057 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc9b69-da89-465e-8af9-67dccb9a3a77 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:33.747545 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc9b69-da89-465e-8af9-67dccb9a3a77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:33.749004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-d5059eb0-c6bd-4441-8ff3-01e289265b08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:26:33.749004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:33.749004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:33.749004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:33.749004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:33.749004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:33.749004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:33.749004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:33.749004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:33.749004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:33.749004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:33.749004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:33.749004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:33.749004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:33.749004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:33.749004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:33.749975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:33.749975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:33.749975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:33.749975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:33.749975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:33.749975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:33.749975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:33.749975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:33.749975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:33.749975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:33.749975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:33.749975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:33.749975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:33.749975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:33.749975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:33.749975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:33.750841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:33.750841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:33.750841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:33.750841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:33.750841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:33.750841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:33.750841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:33.750841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:33.750841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:33.750841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:33.750841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:26:33.750841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:33.750841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 365/743] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:26:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:33.751543 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5059eb0-c6bd-4441-8ff3-01e289265b08 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:33.760631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc9b69-da89-465e-8af9-67dccb9a3a77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:33.762418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc9b69-da89-465e-8af9-67dccb9a3a77 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:33.777609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-a8bc9b69-da89-465e-8af9-67dccb9a3a77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:33.777609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:33.777609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:33.777609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:33.777609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:33.777609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:33.777609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:33.777609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:33.777609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:33.777609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:33.777609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:33.777609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:33.777609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:33.777609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:33.777609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:33.779210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:33.779210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:33.779210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:33.779210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:33.779210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:33.779210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:33.779210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:33.779210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:33.779210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:33.779210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:33.779210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:33.779210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:33.779210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:33.779210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:33.779210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:33.779210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:33.779210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:33.780141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:33.780141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:33.780141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:33.780141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:33.780141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:33.780141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:33.780141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:33.780141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:33.780582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 379/744] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:33.781385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc9b69-da89-465e-8af9-67dccb9a3a77 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:33.811923 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:33.880895 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5059eb0-c6bd-4441-8ff3-01e289265b08 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:33.895862 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc9b69-da89-465e-8af9-67dccb9a3a77 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:33.920463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962060d-daa8-4470-8ab6-b78f04b59a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:33.921162 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962060d-daa8-4470-8ab6-b78f04b59a21 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:33.921451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962060d-daa8-4470-8ab6-b78f04b59a21 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:33.921726 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962060d-daa8-4470-8ab6-b78f04b59a21 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:33.922455 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962060d-daa8-4470-8ab6-b78f04b59a21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:33.922770 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962060d-daa8-4470-8ab6-b78f04b59a21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:33.924142 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962060d-daa8-4470-8ab6-b78f04b59a21 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:33.931420 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22daf8-fdc5-45a4-8341-3e61af09fd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:33.932063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22daf8-fdc5-45a4-8341-3e61af09fd83 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:33.932350 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22daf8-fdc5-45a4-8341-3e61af09fd83 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:33.932645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22daf8-fdc5-45a4-8341-3e61af09fd83 None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:33.933402 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22daf8-fdc5-45a4-8341-3e61af09fd83 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:33.934804 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22daf8-fdc5-45a4-8341-3e61af09fd83 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:33.936961 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 366/745] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:33.941667 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962060d-daa8-4470-8ab6-b78f04b59a21 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:33.961118 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 380/746] 10.222.0.71 () {60 vars in 1141 bytes} [Thu Jul 21 20:26:33 2022] POST /identity/v3/services => generated 240 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:26:33.961118 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22daf8-fdc5-45a4-8341-3e61af09fd83 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:33.981837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:34.065371 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962060d-daa8-4470-8ab6-b78f04b59a21 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:34.089481 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:34.102454 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22daf8-fdc5-45a4-8341-3e61af09fd83 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:34.105036 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80783a86-9ea7-4d57-8cc7-c3a9f15dbfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:34.105770 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80783a86-9ea7-4d57-8cc7-c3a9f15dbfb8 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:34.106598 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80783a86-9ea7-4d57-8cc7-c3a9f15dbfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:34.106875 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80783a86-9ea7-4d57-8cc7-c3a9f15dbfb8 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:34.107652 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80783a86-9ea7-4d57-8cc7-c3a9f15dbfb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:34.107962 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80783a86-9ea7-4d57-8cc7-c3a9f15dbfb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:34.109396 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80783a86-9ea7-4d57-8cc7-c3a9f15dbfb8 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:34.124815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 367/747] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:26:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:34.125844 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80783a86-9ea7-4d57-8cc7-c3a9f15dbfb8 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:34.149837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37622dbc-a3a1-4d17-8de4-99d13c50e755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:34.150202 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37622dbc-a3a1-4d17-8de4-99d13c50e755 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:34.150613 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37622dbc-a3a1-4d17-8de4-99d13c50e755 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:34.150783 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37622dbc-a3a1-4d17-8de4-99d13c50e755 None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:34.151553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37622dbc-a3a1-4d17-8de4-99d13c50e755 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:34.151863 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37622dbc-a3a1-4d17-8de4-99d13c50e755 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:34.154735 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37622dbc-a3a1-4d17-8de4-99d13c50e755 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:34.172048 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 381/748] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:34.172989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37622dbc-a3a1-4d17-8de4-99d13c50e755 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:34.270301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80783a86-9ea7-4d57-8cc7-c3a9f15dbfb8 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:34.285523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:34.317105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37622dbc-a3a1-4d17-8de4-99d13c50e755 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:34.327606 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee4872-1620-4817-b8c9-cde8e7f766f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:34.329952 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee4872-1620-4817-b8c9-cde8e7f766f3 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:34.330673 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee4872-1620-4817-b8c9-cde8e7f766f3 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:34.331916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee4872-1620-4817-b8c9-cde8e7f766f3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:34.347200 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee4872-1620-4817-b8c9-cde8e7f766f3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:34.350251 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee4872-1620-4817-b8c9-cde8e7f766f3 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:34.361328 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294d14e-2782-4a40-ad03-fdb0491c6540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:34.362230 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294d14e-2782-4a40-ad03-fdb0491c6540 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:34.362583 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294d14e-2782-4a40-ad03-fdb0491c6540 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:34.362932 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294d14e-2782-4a40-ad03-fdb0491c6540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:34.370546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-92ee4872-1620-4817-b8c9-cde8e7f766f3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:26:34.370546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:34.370546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:34.370546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:34.370546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:34.370546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:34.370546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:34.370546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:34.370546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:34.370546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:34.370546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:34.370546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:34.370546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:34.370546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:34.370546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:34.370546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:34.371760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:34.371760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:34.371760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:34.371760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:34.371760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:34.371760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:34.371760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:34.371760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:34.371760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:34.371760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:34.371760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:34.371760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:34.371760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:34.371760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:34.371760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:34.371760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:34.371760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:34.371760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:34.372718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:34.372718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:34.372718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:34.372718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:34.372718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:34.372718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:34.372718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:34.372718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:34.372718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:26:34.372718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:34.376806 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 368/749] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:26:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:34.378015 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee4872-1620-4817-b8c9-cde8e7f766f3 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:34.380309 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294d14e-2782-4a40-ad03-fdb0491c6540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:34.382299 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294d14e-2782-4a40-ad03-fdb0491c6540 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:34.385527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:34.404387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-6294d14e-2782-4a40-ad03-fdb0491c6540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:34.404387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:34.404387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:34.404387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:34.404387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:34.404387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:34.404387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:34.404387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:34.404387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:34.404387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:34.404387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:34.404387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:34.404387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:34.404387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:34.404387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:34.405429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:34.405429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:34.405429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:34.405429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:34.405429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:34.405429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:34.405429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:34.405429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:34.405429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:34.405429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:34.405429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:34.405429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:34.405429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:34.405429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:34.405429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:34.405429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:34.405429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:34.406327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:34.406327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:34.406327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:34.406327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:34.408520 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 382/750] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:34.409484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294d14e-2782-4a40-ad03-fdb0491c6540 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:34.497813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee4872-1620-4817-b8c9-cde8e7f766f3 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:34.539044 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294d14e-2782-4a40-ad03-fdb0491c6540 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:34.550040 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30a199-effb-4a7f-898c-0817e3a56740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:34.551706 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30a199-effb-4a7f-898c-0817e3a56740 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:34.552144 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30a199-effb-4a7f-898c-0817e3a56740 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:34.552856 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30a199-effb-4a7f-898c-0817e3a56740 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:34.554218 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30a199-effb-4a7f-898c-0817e3a56740 None admin] RBAC: Adding query filter params (role.id=9a48ab1bf32a45828e9e73599c980dac, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=b94e634e8855486a9ce14d3de9d986fb) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:34.554694 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30a199-effb-4a7f-898c-0817e3a56740 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:34.556868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30a199-effb-4a7f-898c-0817e3a56740 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:34.569573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 369/751] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:26:34 2022] GET /identity/v3/role_assignments?user.id=b94e634e8855486a9ce14d3de9d986fb&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=9a48ab1bf32a45828e9e73599c980dac => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:34.571100 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30a199-effb-4a7f-898c-0817e3a56740 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:34.602460 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986ffa2-6603-47e7-be8a-13aad10dd3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:34.603148 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986ffa2-6603-47e7-be8a-13aad10dd3bc None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:34.603426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986ffa2-6603-47e7-be8a-13aad10dd3bc None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:34.603700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986ffa2-6603-47e7-be8a-13aad10dd3bc None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:34.604556 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986ffa2-6603-47e7-be8a-13aad10dd3bc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:34.604864 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986ffa2-6603-47e7-be8a-13aad10dd3bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:34.610090 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986ffa2-6603-47e7-be8a-13aad10dd3bc None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:34.615982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:34.649124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 383/752] 10.222.0.71 () {56 vars in 1135 bytes} [Thu Jul 21 20:26:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:34.655070 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986ffa2-6603-47e7-be8a-13aad10dd3bc None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:34.716074 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30a199-effb-4a7f-898c-0817e3a56740 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:34.755996 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b72bb-5290-4ef9-bf05-568d87ac7843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:34.756670 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b72bb-5290-4ef9-bf05-568d87ac7843 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:34.757037 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b72bb-5290-4ef9-bf05-568d87ac7843 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:34.757338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b72bb-5290-4ef9-bf05-568d87ac7843 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:34.758078 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b72bb-5290-4ef9-bf05-568d87ac7843 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:34.759169 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b72bb-5290-4ef9-bf05-568d87ac7843 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:34.760295 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b72bb-5290-4ef9-bf05-568d87ac7843 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:34.764036 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:34.776562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 370/753] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:34.784720 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b72bb-5290-4ef9-bf05-568d87ac7843 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:34.869013 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986ffa2-6603-47e7-be8a-13aad10dd3bc None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:34.914905 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783beb05-99b8-47db-a67b-60286b5f1ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:34.915569 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783beb05-99b8-47db-a67b-60286b5f1ce9 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:34.915851 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783beb05-99b8-47db-a67b-60286b5f1ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:34.916126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783beb05-99b8-47db-a67b-60286b5f1ce9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:34.984801 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:34.991143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783beb05-99b8-47db-a67b-60286b5f1ce9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:34.992763 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783beb05-99b8-47db-a67b-60286b5f1ce9 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:35.050724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:35.079120 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b72bb-5290-4ef9-bf05-568d87ac7843 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:35.101756 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 384/754] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:26:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:26:35.107398 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783beb05-99b8-47db-a67b-60286b5f1ce9 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:35.142492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416d24c-62eb-495a-b604-cb525bb41d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:35.143220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416d24c-62eb-495a-b604-cb525bb41d10 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:35.143500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416d24c-62eb-495a-b604-cb525bb41d10 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:35.143772 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416d24c-62eb-495a-b604-cb525bb41d10 None admin] PATH_INFO: `/v3/users/nova` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:35.158887 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416d24c-62eb-495a-b604-cb525bb41d10 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:35.160546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416d24c-62eb-495a-b604-cb525bb41d10 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:35.196857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-4416d24c-62eb-495a-b604-cb525bb41d10 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:26:35.196857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:35.196857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:35.196857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:35.196857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:35.196857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:35.196857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:35.196857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:35.196857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:35.196857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:35.196857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:35.196857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:35.196857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:35.196857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:35.196857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:35.196857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:35.197928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:35.197928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:35.197928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:35.197928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:35.197928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:35.197928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:35.197928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:35.197928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:35.197928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:35.197928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:35.197928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:35.197928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:35.197928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:35.197928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:35.197928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:35.197928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:35.197928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:35.197928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:35.198915 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:35.198915 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:35.198915 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:35.198915 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:35.198915 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:35.198915 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:35.198915 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:35.198915 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:35.198915 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:26:35.198915 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:35.198915 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 371/755] 10.222.0.71 () {56 vars in 1093 bytes} [Thu Jul 21 20:26:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:35.206059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416d24c-62eb-495a-b604-cb525bb41d10 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:35.296729 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783beb05-99b8-47db-a67b-60286b5f1ce9 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:35.335763 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba2f06-890a-4e2b-bf9d-b21b2ac8d251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:35.336766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba2f06-890a-4e2b-bf9d-b21b2ac8d251 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:35.337179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba2f06-890a-4e2b-bf9d-b21b2ac8d251 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:35.337558 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba2f06-890a-4e2b-bf9d-b21b2ac8d251 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:35.343148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416d24c-62eb-495a-b604-cb525bb41d10 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:35.350295 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba2f06-890a-4e2b-bf9d-b21b2ac8d251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:35.351949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba2f06-890a-4e2b-bf9d-b21b2ac8d251 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:35.370972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-a3ba2f06-890a-4e2b-bf9d-b21b2ac8d251 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:35.370972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:35.370972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:35.370972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:35.370972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:35.370972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:35.370972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:35.370972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:35.370972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:35.370972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:35.370972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:35.370972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:35.370972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:35.370972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:35.370972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:35.372201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:35.372201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:35.372201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:35.372201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:35.372201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:35.372201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:35.372201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:35.372201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:35.372201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:35.372201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:35.372201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:35.372201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:35.372201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:35.372201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:35.372201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:35.372201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:35.372201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:35.373434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:35.373434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:35.373434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:35.373434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:35.374904 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 385/756] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:35.376284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad867c9c-88c8-43aa-b3fd-9e98d448959f None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:35.376672 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad867c9c-88c8-43aa-b3fd-9e98d448959f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:35.377048 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad867c9c-88c8-43aa-b3fd-9e98d448959f None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:35.378729 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 386/757] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:35.383848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98d40b-89b7-464b-94ea-e85f94f54f64 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:35.384292 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98d40b-89b7-464b-94ea-e85f94f54f64 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:35.384699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98d40b-89b7-464b-94ea-e85f94f54f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:35.389829 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c6bdf-bf93-482b-aa0c-8aafdf376b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:35.390509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c6bdf-bf93-482b-aa0c-8aafdf376b7f None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:35.390795 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c6bdf-bf93-482b-aa0c-8aafdf376b7f None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:35.391072 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c6bdf-bf93-482b-aa0c-8aafdf376b7f None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:35.391932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7c6bdf-bf93-482b-aa0c-8aafdf376b7f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:35.392245 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7c6bdf-bf93-482b-aa0c-8aafdf376b7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:35.393917 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7c6bdf-bf93-482b-aa0c-8aafdf376b7f None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:35.398771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:35.403202 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:35.407411 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-ea98d40b-89b7-464b-94ea-e85f94f54f64 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:35.408694 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-ea98d40b-89b7-464b-94ea-e85f94f54f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:35.416041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 372/758] 10.222.0.71 () {56 vars in 1133 bytes} [Thu Jul 21 20:26:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:35.417278 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c6bdf-bf93-482b-aa0c-8aafdf376b7f None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:35.488424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 387/759] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:26:35.489574 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98d40b-89b7-464b-94ea-e85f94f54f64 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:35.551538 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c6bdf-bf93-482b-aa0c-8aafdf376b7f None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:35.590010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d612d-51ac-47b2-8d51-448f796e470c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:35.590845 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d612d-51ac-47b2-8d51-448f796e470c None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:35.591248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d612d-51ac-47b2-8d51-448f796e470c None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:35.591623 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d612d-51ac-47b2-8d51-448f796e470c None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:35.592470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d612d-51ac-47b2-8d51-448f796e470c None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:35.592875 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d612d-51ac-47b2-8d51-448f796e470c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:35.594392 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d612d-51ac-47b2-8d51-448f796e470c None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:35.604289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98d40b-89b7-464b-94ea-e85f94f54f64 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:35.612329 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 373/760] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:35.613725 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d612d-51ac-47b2-8d51-448f796e470c None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:35.643701 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c0dcd-9432-4506-86d2-537871baecbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:35.644991 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c0dcd-9432-4506-86d2-537871baecbb None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:35.645468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c0dcd-9432-4506-86d2-537871baecbb None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:35.645903 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c0dcd-9432-4506-86d2-537871baecbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:35.649159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:35.659969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c0dcd-9432-4506-86d2-537871baecbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:35.661774 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c0dcd-9432-4506-86d2-537871baecbb None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:35.680670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-2e1c0dcd-9432-4506-86d2-537871baecbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:35.680670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:35.680670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:35.680670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:35.680670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:35.680670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:35.680670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:35.680670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:35.680670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:35.680670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:35.680670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:35.680670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:35.680670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:35.680670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:35.680670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:35.681878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:35.681878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:35.681878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:35.681878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:35.681878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:35.681878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:35.681878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:35.681878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:35.681878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:35.681878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:35.681878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:35.681878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:35.681878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:35.681878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:35.681878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:35.681878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:35.681878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:35.682900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:35.682900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:35.682900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:35.682900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:35.683138 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:35.685771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 388/761] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:35.687006 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c0dcd-9432-4506-86d2-537871baecbb None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:35.733786 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d612d-51ac-47b2-8d51-448f796e470c None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:35.772920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49959a82-130f-4c12-b6b1-da1eda95b4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:35.773774 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49959a82-130f-4c12-b6b1-da1eda95b4ab None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:35.774063 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49959a82-130f-4c12-b6b1-da1eda95b4ab None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:35.774340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49959a82-130f-4c12-b6b1-da1eda95b4ab None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:35.775009 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49959a82-130f-4c12-b6b1-da1eda95b4ab None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:35.776401 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49959a82-130f-4c12-b6b1-da1eda95b4ab None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:35.791287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-49959a82-130f-4c12-b6b1-da1eda95b4ab None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 20:26:35.791287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:35.791287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:26:35.791287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:26:35.791287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:26:35.791287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:26:35.791287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:26:35.791287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:35.791287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 20:26:35.791287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:35.791287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:35.791287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:35.791287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:35.791287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:35.791287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:35.791287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:35.792305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:35.792305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:35.792305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:35.792305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:35.792305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:35.792305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:35.792305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:35.792305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:26:35.792305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:26:35.792305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:26:35.792305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:26:35.792305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:35.792305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:35.792305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:35.792305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:35.792305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:35.793192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:35.793192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:35.793192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:35.793192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:35.793192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:35.793192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:35.793192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:35.793192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:35.793192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:35.793192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:35.793192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:35.793192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:26:35.793192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:35.793192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 20:26:35.793192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:35.793192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 374/762] 10.222.0.71 () {56 vars in 1129 bytes} [Thu Jul 21 20:26:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:35.794096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49959a82-130f-4c12-b6b1-da1eda95b4ab None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:35.821831 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c0dcd-9432-4506-86d2-537871baecbb None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:35.860590 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad17ec-f349-4135-8083-19b688ebfe69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:35.861591 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad17ec-f349-4135-8083-19b688ebfe69 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:35.861980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad17ec-f349-4135-8083-19b688ebfe69 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:35.862359 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad17ec-f349-4135-8083-19b688ebfe69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:35.872972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad17ec-f349-4135-8083-19b688ebfe69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:35.875216 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad17ec-f349-4135-8083-19b688ebfe69 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:35.893837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-5dad17ec-f349-4135-8083-19b688ebfe69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:35.893837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:35.893837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:35.893837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:35.893837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:35.893837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:35.893837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:35.893837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:35.893837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:35.893837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:35.893837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:35.893837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:35.893837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:35.893837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:35.893837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:35.895297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:35.895297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:35.895297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:35.895297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:35.895297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:35.895297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:35.895297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:35.895297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:35.895297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:35.895297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:35.895297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:35.895297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:35.895297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:35.895297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:35.895297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:35.895297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:35.895297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:35.896855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:35.896855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:35.896855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:35.896855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:35.896855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:35.896855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:35.896855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:35.896855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:35.900587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 389/763] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:35.901621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad17ec-f349-4135-8083-19b688ebfe69 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:35.920813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49959a82-130f-4c12-b6b1-da1eda95b4ab None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:35.942787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:35.961174 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655203b3-b798-4807-8b31-931634c1ecd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:35.962249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655203b3-b798-4807-8b31-931634c1ecd7 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:35.962734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655203b3-b798-4807-8b31-931634c1ecd7 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:35.963233 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655203b3-b798-4807-8b31-931634c1ecd7 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:35.964302 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655203b3-b798-4807-8b31-931634c1ecd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:35.964813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655203b3-b798-4807-8b31-931634c1ecd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:35.967445 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655203b3-b798-4807-8b31-931634c1ecd7 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:35.985134 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:35.993031 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 375/764] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:35.994677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655203b3-b798-4807-8b31-931634c1ecd7 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:36.030451 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad17ec-f349-4135-8083-19b688ebfe69 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:36.077031 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac907df2-59b6-448c-a431-eb9b404d55c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:36.078023 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac907df2-59b6-448c-a431-eb9b404d55c7 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:36.078315 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac907df2-59b6-448c-a431-eb9b404d55c7 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:36.078594 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac907df2-59b6-448c-a431-eb9b404d55c7 None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:36.079323 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac907df2-59b6-448c-a431-eb9b404d55c7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:36.079633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac907df2-59b6-448c-a431-eb9b404d55c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:36.082047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac907df2-59b6-448c-a431-eb9b404d55c7 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:36.087857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 390/765] 10.222.0.71 () {56 vars in 1124 bytes} [Thu Jul 21 20:26:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:36.088715 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac907df2-59b6-448c-a431-eb9b404d55c7 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:36.125442 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655203b3-b798-4807-8b31-931634c1ecd7 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:36.174907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38032320-a38f-4c3d-8f05-9a969c4d1ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:36.175576 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38032320-a38f-4c3d-8f05-9a969c4d1ca0 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:36.175858 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38032320-a38f-4c3d-8f05-9a969c4d1ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:36.176132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38032320-a38f-4c3d-8f05-9a969c4d1ca0 None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:36.176931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38032320-a38f-4c3d-8f05-9a969c4d1ca0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:36.177266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38032320-a38f-4c3d-8f05-9a969c4d1ca0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:36.178697 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38032320-a38f-4c3d-8f05-9a969c4d1ca0 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:36.191941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 376/766] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:36.192858 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38032320-a38f-4c3d-8f05-9a969c4d1ca0 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:36.220519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac907df2-59b6-448c-a431-eb9b404d55c7 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:36.235004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:36.253911 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:36.260671 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e833a-9002-475b-aa5d-6aec771589a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:36.261560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e833a-9002-475b-aa5d-6aec771589a4 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:36.261908 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e833a-9002-475b-aa5d-6aec771589a4 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:36.263036 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e833a-9002-475b-aa5d-6aec771589a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:36.270263 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e833a-9002-475b-aa5d-6aec771589a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:36.271900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e833a-9002-475b-aa5d-6aec771589a4 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:36.287808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-d58e833a-9002-475b-aa5d-6aec771589a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:36.287808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:36.287808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:36.287808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:36.287808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:36.287808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:36.287808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:36.287808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:36.287808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:36.287808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:36.287808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:36.287808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:36.287808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:36.287808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:36.287808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:36.288796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:36.288796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:36.288796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:36.288796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:36.288796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:36.288796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:36.288796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:36.288796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:36.288796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:36.288796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:36.288796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:36.288796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:36.288796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:36.288796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:36.288796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:36.288796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:36.288796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:36.289738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:36.289738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:36.289738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:36.289738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:36.289738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:36.289738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:36.289738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:36.289738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:36.295242 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 391/767] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:36.295395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e833a-9002-475b-aa5d-6aec771589a4 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:36.337295 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38032320-a38f-4c3d-8f05-9a969c4d1ca0 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:36.379183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a61952-66ca-46de-8aee-e8c54874e360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:36.379853 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a61952-66ca-46de-8aee-e8c54874e360 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:36.380168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a61952-66ca-46de-8aee-e8c54874e360 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:36.380675 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a61952-66ca-46de-8aee-e8c54874e360 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:36.381148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a61952-66ca-46de-8aee-e8c54874e360 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:36.381464 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a61952-66ca-46de-8aee-e8c54874e360 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:36.382899 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a61952-66ca-46de-8aee-e8c54874e360 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:36.388447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 377/768] 10.222.0.71 () {56 vars in 1124 bytes} [Thu Jul 21 20:26:36 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:36.393817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a61952-66ca-46de-8aee-e8c54874e360 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:36.406435 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e833a-9002-475b-aa5d-6aec771589a4 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:36.443057 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3a266-1241-4a6f-a82b-1a2a5329c04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:36.443631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3a266-1241-4a6f-a82b-1a2a5329c04c None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:36.443912 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3a266-1241-4a6f-a82b-1a2a5329c04c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:36.444186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3a266-1241-4a6f-a82b-1a2a5329c04c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:36.445015 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3a266-1241-4a6f-a82b-1a2a5329c04c None admin] RBAC: Adding query filter params (role.id=19f5e72c0be44a1197c7288de82e79e8, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=3babb078203647e2b3ef3b2787b84e3b) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:36.445354 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3a266-1241-4a6f-a82b-1a2a5329c04c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:36.446802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3a266-1241-4a6f-a82b-1a2a5329c04c None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:36.461730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 392/769] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:26:36 2022] GET /identity/v3/role_assignments?user.id=3babb078203647e2b3ef3b2787b84e3b&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:36.461730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e39c2-e998-49cb-98e1-1a24579e8ad9 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:36.461730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e39c2-e998-49cb-98e1-1a24579e8ad9 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:36.461730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e39c2-e998-49cb-98e1-1a24579e8ad9 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:36.465008 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 393/770] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:36.466818 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e39c2-e998-49cb-98e1-1a24579e8ad9 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:36.516730 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a61952-66ca-46de-8aee-e8c54874e360 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:36.520555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:36.523328 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:36.561418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778308a7-1876-43cc-90b4-b0adfd640859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:36.562222 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778308a7-1876-43cc-90b4-b0adfd640859 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:36.562599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778308a7-1876-43cc-90b4-b0adfd640859 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:36.563006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778308a7-1876-43cc-90b4-b0adfd640859 None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:36.563901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778308a7-1876-43cc-90b4-b0adfd640859 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:36.564330 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778308a7-1876-43cc-90b4-b0adfd640859 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:36.566114 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778308a7-1876-43cc-90b4-b0adfd640859 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:36.581112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 378/771] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:36.582590 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408d943-4eaa-4855-86cb-14d039934091 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:36.582957 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408d943-4eaa-4855-86cb-14d039934091 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:36.583424 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408d943-4eaa-4855-86cb-14d039934091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:36.603177 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-4408d943-4eaa-4855-86cb-14d039934091 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:36.604651 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-4408d943-4eaa-4855-86cb-14d039934091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:36.617133 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e39c2-e998-49cb-98e1-1a24579e8ad9 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:36.666775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1365c-bd42-411a-b8f6-42a99d808368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:36.667605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1365c-bd42-411a-b8f6-42a99d808368 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:36.667890 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1365c-bd42-411a-b8f6-42a99d808368 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:36.669018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1365c-bd42-411a-b8f6-42a99d808368 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:36.669853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1365c-bd42-411a-b8f6-42a99d808368 None admin] RBAC: Adding query filter params (interface=public, service_id=ed884453171344629d18d624421877fe, region_id=RegionOne) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:36.670171 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1365c-bd42-411a-b8f6-42a99d808368 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:36.671979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1365c-bd42-411a-b8f6-42a99d808368 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:36.683342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 394/772] 10.222.0.71 () {56 vars in 1251 bytes} [Thu Jul 21 20:26:36 2022] GET /identity/v3/endpoints?service_id=ed884453171344629d18d624421877fe&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:36.684254 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1365c-bd42-411a-b8f6-42a99d808368 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:36.691201 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 379/773] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:26:36.692145 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408d943-4eaa-4855-86cb-14d039934091 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:36.807027 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:36.811509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:36.811642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1365c-bd42-411a-b8f6-42a99d808368 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:36.818966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408d943-4eaa-4855-86cb-14d039934091 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:36.852412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edaf99b-1ca5-42fd-8289-c49b21990410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:36.853531 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaf99b-1ca5-42fd-8289-c49b21990410 None admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:36.853877 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaf99b-1ca5-42fd-8289-c49b21990410 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:36.854218 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaf99b-1ca5-42fd-8289-c49b21990410 None admin] PATH_INFO: `/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/6fbc5240f5214aaea4418052b94739e2/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:36.856794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edaf99b-1ca5-42fd-8289-c49b21990410 None admin] RBAC: Authorizing `identity:create_grant(project_id=37e4afa595894a9994a8e6981ebb7c41, user_id=6fbc5240f5214aaea4418052b94739e2, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:36.856912 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef26926-d6fb-48d5-a4e9-5a68fc43391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:36.857695 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef26926-d6fb-48d5-a4e9-5a68fc43391d None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:36.858046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef26926-d6fb-48d5-a4e9-5a68fc43391d None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:36.858395 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef26926-d6fb-48d5-a4e9-5a68fc43391d None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:36.858598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edaf99b-1ca5-42fd-8289-c49b21990410 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:36.859231 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef26926-d6fb-48d5-a4e9-5a68fc43391d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:36.860659 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef26926-d6fb-48d5-a4e9-5a68fc43391d None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:36.877355 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 380/774] 10.222.0.71 () {56 vars in 1087 bytes} [Thu Jul 21 20:26:36 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:36.877355 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef26926-d6fb-48d5-a4e9-5a68fc43391d None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:36.893411 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 395/775] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:26:36 2022] PUT /identity/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/6fbc5240f5214aaea4418052b94739e2/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:26:36.895159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac6107-43ac-4108-8c5a-e66dbbaaff66 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:36.895593 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac6107-43ac-4108-8c5a-e66dbbaaff66 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:36.895983 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac6107-43ac-4108-8c5a-e66dbbaaff66 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:36.897997 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 396/776] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:36.907775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501f1b3-f1f5-4775-8708-47190cfa3cd1 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:36.908212 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501f1b3-f1f5-4775-8708-47190cfa3cd1 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:36.908600 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501f1b3-f1f5-4775-8708-47190cfa3cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:36.951091 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-7501f1b3-f1f5-4775-8708-47190cfa3cd1 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:36.952525 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-7501f1b3-f1f5-4775-8708-47190cfa3cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:36.998277 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef26926-d6fb-48d5-a4e9-5a68fc43391d None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:37.073994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 397/777] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:26:37.082681 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92117d-218f-48ab-ba5a-2855fbc27807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:37.083925 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92117d-218f-48ab-ba5a-2855fbc27807 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:37.084457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92117d-218f-48ab-ba5a-2855fbc27807 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:37.084958 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92117d-218f-48ab-ba5a-2855fbc27807 None admin] PATH_INFO: `/v3/services/network` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:37.086108 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92117d-218f-48ab-ba5a-2855fbc27807 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:37.087932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92117d-218f-48ab-ba5a-2855fbc27807 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:37.101376 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501f1b3-f1f5-4775-8708-47190cfa3cd1 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:37.103573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-ba92117d-218f-48ab-ba5a-2855fbc27807 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 20:26:37.103573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:37.103573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:26:37.103573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:26:37.103573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:26:37.103573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:26:37.103573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:26:37.103573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:37.103573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 20:26:37.103573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:37.103573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:37.103573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:37.103573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:37.103573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:37.103573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:37.103573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:37.103573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:37.104588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:37.104588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:37.104588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:37.104588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:37.104588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:37.104588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:37.104588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:26:37.104588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:26:37.104588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:26:37.104588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:26:37.104588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:37.104588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:37.104588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:37.104588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:37.104588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:37.104588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:37.105493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:37.105493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:37.105493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:37.105493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:37.105493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:37.105493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:37.105493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:37.105493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:37.105493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:37.105493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:37.105493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:26:37.105493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:37.105493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 20:26:37.105493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:37.105493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 381/778] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:36 2022] GET /identity/v3/services/network => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:37.117674 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92117d-218f-48ab-ba5a-2855fbc27807 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:37.150999 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:37.259835 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92117d-218f-48ab-ba5a-2855fbc27807 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:37.269461 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:37.288556 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501f1b3-f1f5-4775-8708-47190cfa3cd1 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:37.303786 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e9805-1152-43eb-b4f8-eb6d82cc2f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:37.304469 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e9805-1152-43eb-b4f8-eb6d82cc2f80 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:37.305441 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e9805-1152-43eb-b4f8-eb6d82cc2f80 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:37.305727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e9805-1152-43eb-b4f8-eb6d82cc2f80 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:37.306465 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e9805-1152-43eb-b4f8-eb6d82cc2f80 None admin] RBAC: Adding query filter params (name=network) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:37.306778 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e9805-1152-43eb-b4f8-eb6d82cc2f80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:37.308182 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e9805-1152-43eb-b4f8-eb6d82cc2f80 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:37.314737 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 382/779] 10.222.0.71 () {56 vars in 1112 bytes} [Thu Jul 21 20:26:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:37.322895 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e9805-1152-43eb-b4f8-eb6d82cc2f80 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:37.368224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a5d80-8b2b-4997-ba1b-da832bc3ebcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:37.369023 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a5d80-8b2b-4997-ba1b-da832bc3ebcf None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:37.369338 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a5d80-8b2b-4997-ba1b-da832bc3ebcf None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:37.369611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a5d80-8b2b-4997-ba1b-da832bc3ebcf None admin] PATH_INFO: `/v3/roles/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:37.392184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a5d80-8b2b-4997-ba1b-da832bc3ebcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:37.410016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-8b4a5d80-8b2b-4997-ba1b-da832bc3ebcf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:37.410016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:37.410016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:37.410016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:37.410016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:37.410016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:37.410016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:37.410016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:37.410016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:37.410016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:37.410016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:37.410016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:37.410016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:37.410016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:37.410016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:37.410016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:37.411478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:37.411478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:37.411478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:37.411478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:37.411478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:37.411478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:37.411478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:37.411478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:37.411478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:37.411478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:37.411478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:37.411478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:37.411478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:37.411478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:37.411478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:37.411478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:37.412337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:37.412337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:37.412337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:37.412337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:37.412337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:37.412337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:37.412337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:37.412337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:37.412337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:37.412337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:37.412337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:37.412337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:37.412337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 398/780] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:26:37.417668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a5d80-8b2b-4997-ba1b-da832bc3ebcf None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:37.434876 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:37.453107 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e9805-1152-43eb-b4f8-eb6d82cc2f80 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:37.504300 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45d651-d0bd-469e-91b1-bcd238ace872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:37.504978 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45d651-d0bd-469e-91b1-bcd238ace872 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:37.505467 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45d651-d0bd-469e-91b1-bcd238ace872 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:37.505998 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45d651-d0bd-469e-91b1-bcd238ace872 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:37.506801 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45d651-d0bd-469e-91b1-bcd238ace872 None admin] RBAC: Adding query filter params (type=network) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:37.507115 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45d651-d0bd-469e-91b1-bcd238ace872 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:37.508527 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45d651-d0bd-469e-91b1-bcd238ace872 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:37.515417 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 383/781] 10.222.0.71 () {56 vars in 1112 bytes} [Thu Jul 21 20:26:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:37.520769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45d651-d0bd-469e-91b1-bcd238ace872 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:37.632078 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a5d80-8b2b-4997-ba1b-da832bc3ebcf None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:37.655330 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45d651-d0bd-469e-91b1-bcd238ace872 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:37.693619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35e849-4e21-4939-a7ca-441c97b0ae39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:37.694732 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35e849-4e21-4939-a7ca-441c97b0ae39 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:37.695314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35e849-4e21-4939-a7ca-441c97b0ae39 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:37.695811 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35e849-4e21-4939-a7ca-441c97b0ae39 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:37.695934 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b8aa1-65e6-4169-9567-e74bc1a9ba2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:37.696545 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b8aa1-65e6-4169-9567-e74bc1a9ba2a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:37.696830 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b8aa1-65e6-4169-9567-e74bc1a9ba2a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:37.697130 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b8aa1-65e6-4169-9567-e74bc1a9ba2a None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:37.697429 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35e849-4e21-4939-a7ca-441c97b0ae39 None admin] RBAC: Adding query filter params (interface=public, service_id=6e903545860a4c68becc58fc10e87028, region_id=RegionOne) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:37.697889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b8aa1-65e6-4169-9567-e74bc1a9ba2a None admin] RBAC: Adding query filter params (name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:37.698005 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35e849-4e21-4939-a7ca-441c97b0ae39 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:37.698201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b8aa1-65e6-4169-9567-e74bc1a9ba2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:37.700208 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b8aa1-65e6-4169-9567-e74bc1a9ba2a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:37.701113 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35e849-4e21-4939-a7ca-441c97b0ae39 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:37.713807 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 384/782] 10.222.0.71 () {56 vars in 1251 bytes} [Thu Jul 21 20:26:37 2022] GET /identity/v3/endpoints?service_id=6e903545860a4c68becc58fc10e87028&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:37.719921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35e849-4e21-4939-a7ca-441c97b0ae39 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:37.725262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 399/783] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:26:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:37.730835 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b8aa1-65e6-4169-9567-e74bc1a9ba2a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:37.755945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:37.757441 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:37.855154 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35e849-4e21-4939-a7ca-441c97b0ae39 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:37.892658 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e168d2-3167-42cb-8b13-a820f4bc1d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:37.893322 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e168d2-3167-42cb-8b13-a820f4bc1d2e None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:37.893604 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e168d2-3167-42cb-8b13-a820f4bc1d2e None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:37.893878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e168d2-3167-42cb-8b13-a820f4bc1d2e None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:37.894895 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e168d2-3167-42cb-8b13-a820f4bc1d2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:37.896272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e168d2-3167-42cb-8b13-a820f4bc1d2e None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:37.902607 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 385/784] 10.222.0.71 () {56 vars in 1087 bytes} [Thu Jul 21 20:26:37 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:37.952456 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b8aa1-65e6-4169-9567-e74bc1a9ba2a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:37.990552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4af54e-0677-4a44-a5b2-5f94063cc9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:37.991348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4af54e-0677-4a44-a5b2-5f94063cc9a2 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:37.991681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4af54e-0677-4a44-a5b2-5f94063cc9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:37.991962 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4af54e-0677-4a44-a5b2-5f94063cc9a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:38.004094 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4af54e-0677-4a44-a5b2-5f94063cc9a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:38.005601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4af54e-0677-4a44-a5b2-5f94063cc9a2 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:38.024802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-0c4af54e-0677-4a44-a5b2-5f94063cc9a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:38.024802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:38.024802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:38.024802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:38.024802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:38.024802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:38.024802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:38.024802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:38.024802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:38.024802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:38.024802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:38.024802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:38.024802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:38.024802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:38.024802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:38.026945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:38.026945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:38.026945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:38.026945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:38.026945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:38.026945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:38.026945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:38.026945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:38.026945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:38.026945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:38.026945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:38.026945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:38.026945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:38.026945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:38.026945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:38.026945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:38.026945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:38.029550 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:38.029550 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:38.029550 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:38.029550 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:38.029550 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 400/785] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:38.038332 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e168d2-3167-42cb-8b13-a820f4bc1d2e None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:38.143775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:38.166539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e168d2-3167-42cb-8b13-a820f4bc1d2e None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:38.203028 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbb23c-7db5-4c36-b799-f15161b2c8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:38.203712 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbb23c-7db5-4c36-b799-f15161b2c8ef None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:38.203991 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbb23c-7db5-4c36-b799-f15161b2c8ef None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:38.204265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbb23c-7db5-4c36-b799-f15161b2c8ef None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:38.205020 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbb23c-7db5-4c36-b799-f15161b2c8ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:38.205356 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbb23c-7db5-4c36-b799-f15161b2c8ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:38.206750 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbb23c-7db5-4c36-b799-f15161b2c8ef None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:38.220349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 386/786] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:38.226243 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4af54e-0677-4a44-a5b2-5f94063cc9a2 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:38.300277 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:38.344666 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4af54e-0677-4a44-a5b2-5f94063cc9a2 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:38.381452 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80faeee8-f37c-4810-87ec-771399238a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:38.382261 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faeee8-f37c-4810-87ec-771399238a17 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:38.382594 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faeee8-f37c-4810-87ec-771399238a17 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:38.382870 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faeee8-f37c-4810-87ec-771399238a17 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:38.390945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80faeee8-f37c-4810-87ec-771399238a17 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:38.392404 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80faeee8-f37c-4810-87ec-771399238a17 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:38.407642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-80faeee8-f37c-4810-87ec-771399238a17 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:26:38.407642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:38.407642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:38.407642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:38.407642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:38.407642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:38.407642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:38.407642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:38.407642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:38.407642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:38.407642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:38.407642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:38.407642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:38.407642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:38.407642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:38.407642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:38.409925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:38.409925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:38.409925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:38.409925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:38.409925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:38.409925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:38.409925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:38.409925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:38.409925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:38.409925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:38.409925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:38.409925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:38.409925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:38.409925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:38.409925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:38.409925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:38.409925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:38.409925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:38.412521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:38.412521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:38.412521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:38.412521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:38.412521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:38.412521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:38.412521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:38.412521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:38.412521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:26:38.412521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:38.412521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 401/787] 10.222.0.71 () {56 vars in 1117 bytes} [Thu Jul 21 20:26:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:38.419855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbb23c-7db5-4c36-b799-f15161b2c8ef None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:38.576165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbb23c-7db5-4c36-b799-f15161b2c8ef None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:38.614489 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478bd808-b77c-4a2b-86db-3b85e0e758c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:38.615166 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478bd808-b77c-4a2b-86db-3b85e0e758c2 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:38.615447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478bd808-b77c-4a2b-86db-3b85e0e758c2 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:38.615724 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478bd808-b77c-4a2b-86db-3b85e0e758c2 None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:38.616639 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478bd808-b77c-4a2b-86db-3b85e0e758c2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:38.616951 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478bd808-b77c-4a2b-86db-3b85e0e758c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:38.618465 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478bd808-b77c-4a2b-86db-3b85e0e758c2 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:38.629452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:38.639710 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 387/788] 10.222.0.71 () {56 vars in 1149 bytes} [Thu Jul 21 20:26:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:38.648447 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80faeee8-f37c-4810-87ec-771399238a17 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:38.756504 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80faeee8-f37c-4810-87ec-771399238a17 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:38.780958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:38.792027 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1e716-23eb-41fc-a71f-810eb10f95cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:38.792870 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1e716-23eb-41fc-a71f-810eb10f95cb None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:38.793241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1e716-23eb-41fc-a71f-810eb10f95cb None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:38.793519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1e716-23eb-41fc-a71f-810eb10f95cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:38.804554 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f1e716-23eb-41fc-a71f-810eb10f95cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:38.806062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f1e716-23eb-41fc-a71f-810eb10f95cb None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:38.824582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-16f1e716-23eb-41fc-a71f-810eb10f95cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:38.824582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:38.824582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:38.824582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:38.824582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:38.824582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:38.824582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:38.824582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:38.824582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:38.824582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:38.824582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:38.824582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:38.824582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:38.824582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:38.824582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:38.826654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:38.826654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:38.826654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:38.826654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:38.826654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:38.826654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:38.826654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:38.826654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:38.826654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:38.826654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:38.826654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:38.826654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:38.826654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:38.826654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:38.826654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:38.826654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:38.826654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:38.827810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:38.827810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:38.827810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:38.827810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:38.827810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 402/789] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:38.838329 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478bd808-b77c-4a2b-86db-3b85e0e758c2 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:38.871111 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd992b2-9828-45e1-bbcf-e3e65f31ca4a None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:38.872047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd992b2-9828-45e1-bbcf-e3e65f31ca4a None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:38.872255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd992b2-9828-45e1-bbcf-e3e65f31ca4a None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:38.873856 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 403/790] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:38.878925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee65754-2c21-45dd-8cad-1be2e0ff935e None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:38.879231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee65754-2c21-45dd-8cad-1be2e0ff935e None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:38.879514 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee65754-2c21-45dd-8cad-1be2e0ff935e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:38.898343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-3ee65754-2c21-45dd-8cad-1be2e0ff935e None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:38.899675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3ee65754-2c21-45dd-8cad-1be2e0ff935e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:39.017928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478bd808-b77c-4a2b-86db-3b85e0e758c2 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:39.023049 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 404/791] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:26:39.056322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee65754-2c21-45dd-8cad-1be2e0ff935e None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:39.067949 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14bb8a-af79-4af7-827a-49fe01545d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:39.068631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14bb8a-af79-4af7-827a-49fe01545d25 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:39.068911 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14bb8a-af79-4af7-827a-49fe01545d25 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:39.069206 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14bb8a-af79-4af7-827a-49fe01545d25 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:39.069942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14bb8a-af79-4af7-827a-49fe01545d25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:39.070249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14bb8a-af79-4af7-827a-49fe01545d25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:39.076847 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14bb8a-af79-4af7-827a-49fe01545d25 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:39.103894 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 388/792] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:39.107363 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14bb8a-af79-4af7-827a-49fe01545d25 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:39.226023 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee65754-2c21-45dd-8cad-1be2e0ff935e None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:39.246050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:39.248042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:39.271019 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3646f-a203-4c1b-b43a-5c387e38c19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:39.272298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3646f-a203-4c1b-b43a-5c387e38c19d None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:39.273096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3646f-a203-4c1b-b43a-5c387e38c19d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:39.274203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3646f-a203-4c1b-b43a-5c387e38c19d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:39.298997 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3646f-a203-4c1b-b43a-5c387e38c19d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:39.314010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-c0a3646f-a203-4c1b-b43a-5c387e38c19d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:26:39.314010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:39.314010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:39.314010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:39.314010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:39.314010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:39.314010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:39.314010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:39.314010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:39.314010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:39.314010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:39.314010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:39.314010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:39.314010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:39.314010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:39.314010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:39.314971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:39.314971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:39.314971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:39.314971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:39.314971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:39.314971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:39.314971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:39.314971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:39.314971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:39.314971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:39.314971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:39.314971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:39.314971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:39.314971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:39.314971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:39.314971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:39.315809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:39.315809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:39.315809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:39.315809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:39.315809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:39.315809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:39.315809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:39.315809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:39.315809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:39.315809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:39.315809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:26:39.315809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:39.318749 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 405/793] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:26:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:39.325869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3646f-a203-4c1b-b43a-5c387e38c19d None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:39.334500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14bb8a-af79-4af7-827a-49fe01545d25 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:39.379060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a42bd-f0b2-4b06-a18d-3060da0c1bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:39.380254 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a42bd-f0b2-4b06-a18d-3060da0c1bba None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:39.380734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a42bd-f0b2-4b06-a18d-3060da0c1bba None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:39.381240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a42bd-f0b2-4b06-a18d-3060da0c1bba None admin] PATH_INFO: `/v3/projects/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:39.397737 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a42bd-f0b2-4b06-a18d-3060da0c1bba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:39.399578 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a42bd-f0b2-4b06-a18d-3060da0c1bba None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:39.432179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-343a42bd-f0b2-4b06-a18d-3060da0c1bba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:39.432179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:39.432179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:39.432179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:39.432179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:39.432179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:39.432179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:39.432179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:39.432179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:39.432179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:39.432179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:39.432179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:39.432179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:39.432179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:39.432179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:39.433165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:39.433165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:39.433165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:39.433165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:39.433165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:39.433165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:39.433165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:39.433165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:39.433165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:39.433165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:39.433165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:39.433165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:39.433165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:39.433165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:39.433165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:39.433165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:39.433165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:39.434677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:39.434677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:39.434677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:39.434677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:39.434677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:39.434677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:39.434677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:39.434677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:39.441622 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 389/794] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:39.445186 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a42bd-f0b2-4b06-a18d-3060da0c1bba None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:39.536491 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3646f-a203-4c1b-b43a-5c387e38c19d None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:39.567843 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a42bd-f0b2-4b06-a18d-3060da0c1bba None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:39.573256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b5536-d70f-4528-9bab-00ffb3d4bbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:39.573935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b5536-d70f-4528-9bab-00ffb3d4bbd4 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:39.574314 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b5536-d70f-4528-9bab-00ffb3d4bbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:39.574685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b5536-d70f-4528-9bab-00ffb3d4bbd4 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:39.575544 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b5536-d70f-4528-9bab-00ffb3d4bbd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:39.575855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b5536-d70f-4528-9bab-00ffb3d4bbd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:39.577367 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b5536-d70f-4528-9bab-00ffb3d4bbd4 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:39.587225 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 406/795] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:26:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:39.588660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea133b2-1b88-4b04-9d7c-8bbe0c569dc3 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:39.588947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea133b2-1b88-4b04-9d7c-8bbe0c569dc3 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:39.589262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea133b2-1b88-4b04-9d7c-8bbe0c569dc3 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:39.591257 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 407/796] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:39.592434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:39.594183 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea133b2-1b88-4b04-9d7c-8bbe0c569dc3 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:39.613478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd9e54-8c94-4ea4-9f66-375e06d63384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:39.614144 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd9e54-8c94-4ea4-9f66-375e06d63384 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:39.614422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd9e54-8c94-4ea4-9f66-375e06d63384 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:39.614693 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd9e54-8c94-4ea4-9f66-375e06d63384 None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:39.615484 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd9e54-8c94-4ea4-9f66-375e06d63384 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:39.615789 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd9e54-8c94-4ea4-9f66-375e06d63384 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:39.617253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd9e54-8c94-4ea4-9f66-375e06d63384 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:39.627892 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:39.635316 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 390/797] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:39.635316 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf08d9-1562-4af6-ad64-166fc14b0283 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:39.635316 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf08d9-1562-4af6-ad64-166fc14b0283 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:39.635316 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf08d9-1562-4af6-ad64-166fc14b0283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:39.650837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-49bf08d9-1562-4af6-ad64-166fc14b0283 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:39.652397 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-49bf08d9-1562-4af6-ad64-166fc14b0283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:39.745516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 391/798] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:26:39.745707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea133b2-1b88-4b04-9d7c-8bbe0c569dc3 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:39.745789 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1294b-8c48-4c0c-96be-33925b63fe92 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:39.745789 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1294b-8c48-4c0c-96be-33925b63fe92 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:39.745789 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1294b-8c48-4c0c-96be-33925b63fe92 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:39.745789 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 392/799] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:39.745789 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1294b-8c48-4c0c-96be-33925b63fe92 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:39.801478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf584f6-15df-425a-a7bf-03d881eb65c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:39.802266 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf584f6-15df-425a-a7bf-03d881eb65c7 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:39.802552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf584f6-15df-425a-a7bf-03d881eb65c7 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:39.802823 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf584f6-15df-425a-a7bf-03d881eb65c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:39.814561 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf584f6-15df-425a-a7bf-03d881eb65c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:39.816018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf584f6-15df-425a-a7bf-03d881eb65c7 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:39.836005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-dcf584f6-15df-425a-a7bf-03d881eb65c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:39.836005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:39.836005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:39.836005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:39.836005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:39.836005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:39.836005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:39.836005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:39.836005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:39.836005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:39.836005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:39.836005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:39.836005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:39.836005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:39.836005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:39.837009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:39.837009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:39.837009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:39.837009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:39.837009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:39.837009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:39.837009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:39.837009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:39.837009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:39.837009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:39.837009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:39.837009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:39.837009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:39.837009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:39.837009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:39.837009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:39.837009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:39.837982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:39.837982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:39.837982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:39.837982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:39.837982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 408/800] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:39.839129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70980788-254b-4ce0-98dc-1f4dd3ec9305 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:39.839129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70980788-254b-4ce0-98dc-1f4dd3ec9305 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:39.839384 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70980788-254b-4ce0-98dc-1f4dd3ec9305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:39.857720 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-70980788-254b-4ce0-98dc-1f4dd3ec9305 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:39.858934 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-70980788-254b-4ce0-98dc-1f4dd3ec9305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:39.872599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1294b-8c48-4c0c-96be-33925b63fe92 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:39.881797 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:39.912949 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981b4f8-f420-408a-9b09-4163078abc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:39.913875 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981b4f8-f420-408a-9b09-4163078abc33 None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:39.914155 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981b4f8-f420-408a-9b09-4163078abc33 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:39.914426 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981b4f8-f420-408a-9b09-4163078abc33 None admin] PATH_INFO: `/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/b94e634e8855486a9ce14d3de9d986fb/roles/9a48ab1bf32a45828e9e73599c980dac` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:39.933536 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981b4f8-f420-408a-9b09-4163078abc33 None admin] RBAC: Authorizing `identity:create_grant(project_id=37e4afa595894a9994a8e6981ebb7c41, user_id=b94e634e8855486a9ce14d3de9d986fb, role_id=9a48ab1bf32a45828e9e73599c980dac)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:39.935558 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981b4f8-f420-408a-9b09-4163078abc33 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:39.941420 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 409/801] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:26:39.951452 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70980788-254b-4ce0-98dc-1f4dd3ec9305 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:39.957757 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 393/802] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:26:39 2022] PUT /identity/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/b94e634e8855486a9ce14d3de9d986fb/roles/9a48ab1bf32a45828e9e73599c980dac => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:26:39.958685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981b4f8-f420-408a-9b09-4163078abc33 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:39.971931 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:40.094615 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70980788-254b-4ce0-98dc-1f4dd3ec9305 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:40.119695 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981b4f8-f420-408a-9b09-4163078abc33 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:40.139183 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fab3dc-fd26-463a-81ac-e8b958114729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:40.140002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fab3dc-fd26-463a-81ac-e8b958114729 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:40.140286 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fab3dc-fd26-463a-81ac-e8b958114729 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:40.140598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fab3dc-fd26-463a-81ac-e8b958114729 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:40.156044 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fab3dc-fd26-463a-81ac-e8b958114729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:40.164345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-c2fab3dc-fd26-463a-81ac-e8b958114729 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:26:40.164345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:40.164345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:40.164345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:40.164345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:40.164345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:40.164345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:40.164345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:40.164345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:40.164345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:40.164345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:40.164345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:40.164345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:40.164345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:40.164345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:40.164345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:40.166040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:40.166040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:40.166040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:40.166040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:40.166040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:40.166040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:40.166040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:40.166040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:40.166040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:40.166040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:40.166040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:40.166040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:40.166040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:40.166040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:40.166040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:40.166040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:40.166938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:40.166938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:40.166938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:40.166938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:40.166938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:40.166938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:40.166938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:40.166938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:40.166938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:40.166938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:40.166938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:26:40.166938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:40.166938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 410/803] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:26:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:40.166938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fab3dc-fd26-463a-81ac-e8b958114729 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:40.175424 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0a173-d5e6-4a3b-ae82-7410152c7402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:40.176107 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0a173-d5e6-4a3b-ae82-7410152c7402 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:40.176496 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0a173-d5e6-4a3b-ae82-7410152c7402 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:40.176682 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0a173-d5e6-4a3b-ae82-7410152c7402 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:40.178944 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0a173-d5e6-4a3b-ae82-7410152c7402 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:40.178944 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0a173-d5e6-4a3b-ae82-7410152c7402 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:40.179312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0a173-d5e6-4a3b-ae82-7410152c7402 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:40.192483 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 394/804] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:40.193390 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0a173-d5e6-4a3b-ae82-7410152c7402 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:40.214281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:40.258161 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:40.298201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fab3dc-fd26-463a-81ac-e8b958114729 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:40.333556 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0a173-d5e6-4a3b-ae82-7410152c7402 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:40.341150 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6b361-58d2-4183-8e69-51a08fcf6e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:40.342369 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6b361-58d2-4183-8e69-51a08fcf6e92 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:40.342859 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6b361-58d2-4183-8e69-51a08fcf6e92 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:40.343337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6b361-58d2-4183-8e69-51a08fcf6e92 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:40.344297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6b361-58d2-4183-8e69-51a08fcf6e92 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:40.346124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6b361-58d2-4183-8e69-51a08fcf6e92 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:40.364798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-20c6b361-58d2-4183-8e69-51a08fcf6e92 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 20:26:40.364798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:40.364798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:26:40.364798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:26:40.364798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:26:40.364798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:26:40.364798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:26:40.364798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:40.364798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 20:26:40.364798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:40.364798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:40.364798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:40.364798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:40.364798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:40.364798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:40.364798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:40.365872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:40.365872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:40.365872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:40.365872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:40.365872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:40.365872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:40.365872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:40.365872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:26:40.365872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:26:40.365872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:26:40.365872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:26:40.365872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:40.365872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:40.365872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:40.365872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:40.365872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:40.366748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:40.366748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:40.366748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:40.366748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:40.366748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:40.366748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:40.366748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:40.366748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:40.366748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:40.366748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:40.366748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:40.366748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:26:40.366748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:40.366748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 20:26:40.366748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:40.375543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 411/805] 10.222.0.71 () {56 vars in 1129 bytes} [Thu Jul 21 20:26:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:40.375543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6b361-58d2-4183-8e69-51a08fcf6e92 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:40.381140 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89afba-8c79-4040-a28d-25225bda4025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:40.382184 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89afba-8c79-4040-a28d-25225bda4025 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:40.382677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89afba-8c79-4040-a28d-25225bda4025 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:40.383178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89afba-8c79-4040-a28d-25225bda4025 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:40.384164 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89afba-8c79-4040-a28d-25225bda4025 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:40.384685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89afba-8c79-4040-a28d-25225bda4025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:40.388127 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89afba-8c79-4040-a28d-25225bda4025 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:40.404632 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 395/806] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:26:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:40.405988 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89afba-8c79-4040-a28d-25225bda4025 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:40.515727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:40.517152 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6b361-58d2-4183-8e69-51a08fcf6e92 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:40.548797 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89afba-8c79-4040-a28d-25225bda4025 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:40.562288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ba88b-0d20-4ecb-bcdc-a2ee191becf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:40.563755 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ba88b-0d20-4ecb-bcdc-a2ee191becf3 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:40.563755 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ba88b-0d20-4ecb-bcdc-a2ee191becf3 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:40.563755 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ba88b-0d20-4ecb-bcdc-a2ee191becf3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:40.566209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:40.579365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685ba88b-0d20-4ecb-bcdc-a2ee191becf3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:40.582133 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685ba88b-0d20-4ecb-bcdc-a2ee191becf3 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:40.600466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-685ba88b-0d20-4ecb-bcdc-a2ee191becf3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:26:40.600466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:40.600466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:40.600466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:40.600466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:40.600466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:40.600466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:40.600466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:40.600466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:40.600466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:40.600466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:40.600466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:40.600466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:40.600466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:40.600466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:40.600466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:40.601492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:40.601492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:40.601492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:40.601492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:40.601492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:40.601492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:40.601492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:40.601492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:40.601492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:40.601492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:40.601492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:40.601492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:40.601492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:40.601492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:40.601492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:40.601492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:40.601492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:40.601492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:40.602484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:40.602484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:40.602484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:40.602484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:40.602484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:40.602484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:40.602484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:40.602484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:40.602484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:26:40.602484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:40.603961 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060196a-28b4-42bd-a1b4-f744990fc981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:40.604914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060196a-28b4-42bd-a1b4-f744990fc981 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:40.605434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 412/807] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:26:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:40.606134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060196a-28b4-42bd-a1b4-f744990fc981 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:40.606614 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060196a-28b4-42bd-a1b4-f744990fc981 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:40.606882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ba88b-0d20-4ecb-bcdc-a2ee191becf3 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:40.607895 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060196a-28b4-42bd-a1b4-f744990fc981 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:40.608433 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060196a-28b4-42bd-a1b4-f744990fc981 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:40.610380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060196a-28b4-42bd-a1b4-f744990fc981 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:40.621139 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 396/808] 10.222.0.71 () {56 vars in 1124 bytes} [Thu Jul 21 20:26:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:40.621139 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060196a-28b4-42bd-a1b4-f744990fc981 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:40.740075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ba88b-0d20-4ecb-bcdc-a2ee191becf3 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:40.744369 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060196a-28b4-42bd-a1b4-f744990fc981 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:40.779394 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641dff5-2195-4b5f-93dd-033e425d3121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:40.780194 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641dff5-2195-4b5f-93dd-033e425d3121 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:40.780475 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641dff5-2195-4b5f-93dd-033e425d3121 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:40.780753 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641dff5-2195-4b5f-93dd-033e425d3121 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:40.783364 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad15180-3e4f-49f5-82ff-b42541c78237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:40.784133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad15180-3e4f-49f5-82ff-b42541c78237 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:40.784454 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad15180-3e4f-49f5-82ff-b42541c78237 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:40.784939 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad15180-3e4f-49f5-82ff-b42541c78237 None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:40.785980 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad15180-3e4f-49f5-82ff-b42541c78237 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:40.786289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad15180-3e4f-49f5-82ff-b42541c78237 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:40.787761 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad15180-3e4f-49f5-82ff-b42541c78237 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:40.794825 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641dff5-2195-4b5f-93dd-033e425d3121 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:40.796593 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641dff5-2195-4b5f-93dd-033e425d3121 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:40.811444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:40.822848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-0641dff5-2195-4b5f-93dd-033e425d3121 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:40.822848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:40.822848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:40.822848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:40.822848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:40.822848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:40.822848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:40.822848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:40.822848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:40.822848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:40.822848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:40.822848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:40.822848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:40.822848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:40.822848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:40.824740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:40.824740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:40.824740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:40.824740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:40.824740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:40.824740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:40.824740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:40.824740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:40.824740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:40.824740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:40.824740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:40.824740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:40.824740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:40.824740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:40.824740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:40.824740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:40.824740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:40.826540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:40.826540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:40.826540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:40.826540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:40.826540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 413/809] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:40.826540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641dff5-2195-4b5f-93dd-033e425d3121 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:40.829646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 397/810] 10.222.0.71 () {56 vars in 1135 bytes} [Thu Jul 21 20:26:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:40.832522 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b410f40-780c-483a-ac69-32fe2959dc1d None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:40.833294 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b410f40-780c-483a-ac69-32fe2959dc1d None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:40.834232 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b410f40-780c-483a-ac69-32fe2959dc1d None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:40.837771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 398/811] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:40.842002 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b410f40-780c-483a-ac69-32fe2959dc1d None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:40.889961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:40.954623 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641dff5-2195-4b5f-93dd-033e425d3121 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:40.989765 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b410f40-780c-483a-ac69-32fe2959dc1d None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:40.997159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bffaa9-0469-4166-87c3-732a189a6d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:40.997833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bffaa9-0469-4166-87c3-732a189a6d12 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:40.998119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bffaa9-0469-4166-87c3-732a189a6d12 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:40.998395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bffaa9-0469-4166-87c3-732a189a6d12 None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:40.999116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bffaa9-0469-4166-87c3-732a189a6d12 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:40.999424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bffaa9-0469-4166-87c3-732a189a6d12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:41.000820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bffaa9-0469-4166-87c3-732a189a6d12 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:41.012670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 414/812] 10.222.0.71 () {56 vars in 1124 bytes} [Thu Jul 21 20:26:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:41.012670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bffaa9-0469-4166-87c3-732a189a6d12 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:41.050889 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628934e1-488c-4969-8f78-f233d697ac7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:41.052181 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628934e1-488c-4969-8f78-f233d697ac7b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:41.052645 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628934e1-488c-4969-8f78-f233d697ac7b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:41.053090 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628934e1-488c-4969-8f78-f233d697ac7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:41.076775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628934e1-488c-4969-8f78-f233d697ac7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:41.078616 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628934e1-488c-4969-8f78-f233d697ac7b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:41.101170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-628934e1-488c-4969-8f78-f233d697ac7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:41.101170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:41.101170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:41.101170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:41.101170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:41.101170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:41.101170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:41.101170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:41.101170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:41.101170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:41.101170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:41.101170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:41.101170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:41.101170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:41.101170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:41.102195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:41.102195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:41.102195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:41.102195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:41.102195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:41.102195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:41.102195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:41.102195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:41.102195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:41.102195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:41.102195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:41.102195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:41.102195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:41.102195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:41.102195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:41.102195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:41.102195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:41.103092 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:41.103092 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:41.103092 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:41.103092 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:41.107848 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 399/813] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:41.109155 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bc383-3731-4a43-b40a-c4e66bef0bf7 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:41.109453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bc383-3731-4a43-b40a-c4e66bef0bf7 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:41.109736 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bc383-3731-4a43-b40a-c4e66bef0bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:41.129047 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-b19bc383-3731-4a43-b40a-c4e66bef0bf7 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:41.130342 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-b19bc383-3731-4a43-b40a-c4e66bef0bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:41.147009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bffaa9-0469-4166-87c3-732a189a6d12 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:41.159997 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:41.188586 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:41.189767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb201a-dee7-49e3-82e1-ea82401d74ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:41.190160 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb201a-dee7-49e3-82e1-ea82401d74ba None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:41.190582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb201a-dee7-49e3-82e1-ea82401d74ba None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:41.190863 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb201a-dee7-49e3-82e1-ea82401d74ba None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:41.191818 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb201a-dee7-49e3-82e1-ea82401d74ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:41.192128 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb201a-dee7-49e3-82e1-ea82401d74ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:41.194242 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb201a-dee7-49e3-82e1-ea82401d74ba None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:41.208288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 415/814] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:41.209165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb201a-dee7-49e3-82e1-ea82401d74ba None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:41.221641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 400/815] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:26:41.223029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bc383-3731-4a43-b40a-c4e66bef0bf7 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:41.334703 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb201a-dee7-49e3-82e1-ea82401d74ba None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:41.364331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bc383-3731-4a43-b40a-c4e66bef0bf7 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:41.376757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230e2a4-568a-4b8f-868e-c4129882b186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:41.377446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230e2a4-568a-4b8f-868e-c4129882b186 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:41.377728 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230e2a4-568a-4b8f-868e-c4129882b186 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:41.378050 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230e2a4-568a-4b8f-868e-c4129882b186 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:41.378758 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230e2a4-568a-4b8f-868e-c4129882b186 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:41.380162 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230e2a4-568a-4b8f-868e-c4129882b186 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:41.411400 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 416/816] 10.222.0.71 () {60 vars in 1144 bytes} [Thu Jul 21 20:26:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:26:41.411400 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230e2a4-568a-4b8f-868e-c4129882b186 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:41.432097 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533c255-7c6f-4668-9af1-724b3c9ecd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:41.433450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533c255-7c6f-4668-9af1-724b3c9ecd19 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:41.434064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533c255-7c6f-4668-9af1-724b3c9ecd19 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:41.434540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533c255-7c6f-4668-9af1-724b3c9ecd19 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:41.436260 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533c255-7c6f-4668-9af1-724b3c9ecd19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:41.436734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533c255-7c6f-4668-9af1-724b3c9ecd19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:41.438836 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533c255-7c6f-4668-9af1-724b3c9ecd19 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:41.448588 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:41.456200 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 401/817] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:41.457706 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533c255-7c6f-4668-9af1-724b3c9ecd19 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:41.530432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:41.568156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230e2a4-568a-4b8f-868e-c4129882b186 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:41.619132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533c255-7c6f-4668-9af1-724b3c9ecd19 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:41.635193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de68bd-0ff3-4a27-acd3-08048ee8247b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:41.636046 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de68bd-0ff3-4a27-acd3-08048ee8247b None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:41.636324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de68bd-0ff3-4a27-acd3-08048ee8247b None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:41.637248 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de68bd-0ff3-4a27-acd3-08048ee8247b None admin] PATH_INFO: `/v3/users/nova` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:41.650284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de68bd-0ff3-4a27-acd3-08048ee8247b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:41.654119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de68bd-0ff3-4a27-acd3-08048ee8247b None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:41.677745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-f9de68bd-0ff3-4a27-acd3-08048ee8247b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:26:41.677745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:41.677745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:41.677745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:41.677745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:41.677745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:41.677745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:41.677745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:41.677745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:41.677745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:41.677745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:41.677745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:41.677745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:41.677745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:41.677745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:41.677745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:41.679675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:41.679675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:41.679675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:41.679675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:41.679675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:41.679675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:41.679675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:41.679675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:41.679675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:41.679675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:41.679675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:41.679675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:41.679675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:41.679675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:41.679675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:41.679675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:41.679675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:41.679675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:41.681660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:41.681660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:41.681660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:41.681660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:41.681660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:41.681660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:41.681660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:41.681660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:41.681660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:26:41.681660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:41.681660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 417/818] 10.222.0.71 () {56 vars in 1093 bytes} [Thu Jul 21 20:26:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:41.681660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de68bd-0ff3-4a27-acd3-08048ee8247b None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:41.684612 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26296112-d605-4f96-a45d-40d09ef1a693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:41.685618 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26296112-d605-4f96-a45d-40d09ef1a693 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:41.685909 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26296112-d605-4f96-a45d-40d09ef1a693 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:41.686175 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26296112-d605-4f96-a45d-40d09ef1a693 None admin] PATH_INFO: `/v3/services/network` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:41.686860 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26296112-d605-4f96-a45d-40d09ef1a693 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:41.688303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26296112-d605-4f96-a45d-40d09ef1a693 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:41.703520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-26296112-d605-4f96-a45d-40d09ef1a693 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 20:26:41.703520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:41.703520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:26:41.703520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:26:41.703520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:26:41.703520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:26:41.703520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:26:41.703520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:41.703520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 20:26:41.703520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:41.703520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:41.703520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:41.703520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:41.703520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:41.703520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:41.703520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:41.703520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:41.705896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:41.705896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:41.705896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:41.705896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:41.705896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:41.705896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:41.705896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:26:41.705896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:26:41.705896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:26:41.705896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:26:41.705896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:41.705896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:41.705896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:41.705896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:41.705896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:41.705896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:41.707863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:41.707863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:41.707863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:41.707863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:41.707863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:41.707863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:41.707863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:41.707863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:41.707863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:41.707863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:41.707863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:26:41.707863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:41.707863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 20:26:41.707863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:41.707863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 402/819] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:41 2022] GET /identity/v3/services/network => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:41.709796 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26296112-d605-4f96-a45d-40d09ef1a693 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:41.821903 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:41.842423 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de68bd-0ff3-4a27-acd3-08048ee8247b None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:41.888291 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bf0cf-94da-4ed4-8611-be86182465a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:41.889432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bf0cf-94da-4ed4-8611-be86182465a6 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:41.889805 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bf0cf-94da-4ed4-8611-be86182465a6 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:41.890479 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bf0cf-94da-4ed4-8611-be86182465a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:41.899471 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bf0cf-94da-4ed4-8611-be86182465a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:41.901681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bf0cf-94da-4ed4-8611-be86182465a6 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:41.921794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-951bf0cf-94da-4ed4-8611-be86182465a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:41.921794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:41.921794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:41.921794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:41.921794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:41.921794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:41.921794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:41.921794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:41.921794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:41.921794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:41.921794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:41.921794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:41.921794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:41.921794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:41.921794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:41.922782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:41.922782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:41.922782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:41.922782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:41.922782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:41.922782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:41.922782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:41.922782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:41.922782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:41.922782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:41.922782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:41.922782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:41.922782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:41.922782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:41.922782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:41.922782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:41.922782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:41.923897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:41.923897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:41.923897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:41.923897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:41.923897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:41.923897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:41.923897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:41.923897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:41.923897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 418/820] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:41.924555 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bf0cf-94da-4ed4-8611-be86182465a6 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:41.955741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26296112-d605-4f96-a45d-40d09ef1a693 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:41.958292 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:41.999301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed3895-4901-4a6c-85e0-4207344e1014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:42.000723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed3895-4901-4a6c-85e0-4207344e1014 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:42.001222 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed3895-4901-4a6c-85e0-4207344e1014 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:42.001761 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed3895-4901-4a6c-85e0-4207344e1014 None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:42.002867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed3895-4901-4a6c-85e0-4207344e1014 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:42.003384 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed3895-4901-4a6c-85e0-4207344e1014 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:42.005016 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed3895-4901-4a6c-85e0-4207344e1014 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:42.030516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 403/821] 10.222.0.71 () {56 vars in 1133 bytes} [Thu Jul 21 20:26:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:42.031755 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed3895-4901-4a6c-85e0-4207344e1014 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:42.075267 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bf0cf-94da-4ed4-8611-be86182465a6 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:42.115212 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359d16a-b9b7-4e41-98f2-73e7c0d238b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:42.115895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359d16a-b9b7-4e41-98f2-73e7c0d238b7 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:42.116181 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359d16a-b9b7-4e41-98f2-73e7c0d238b7 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:42.116459 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359d16a-b9b7-4e41-98f2-73e7c0d238b7 None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:42.117224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359d16a-b9b7-4e41-98f2-73e7c0d238b7 None admin] RBAC: Adding query filter params (name=network) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:42.117538 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359d16a-b9b7-4e41-98f2-73e7c0d238b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:42.118961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359d16a-b9b7-4e41-98f2-73e7c0d238b7 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:42.125314 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 419/822] 10.222.0.71 () {56 vars in 1112 bytes} [Thu Jul 21 20:26:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:42.126166 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359d16a-b9b7-4e41-98f2-73e7c0d238b7 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:42.139621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:42.185337 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed3895-4901-4a6c-85e0-4207344e1014 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:42.231018 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d707e70-d9b1-49b7-8fdd-99c5075979d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:42.231618 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d707e70-d9b1-49b7-8fdd-99c5075979d5 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:42.231899 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d707e70-d9b1-49b7-8fdd-99c5075979d5 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:42.232176 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d707e70-d9b1-49b7-8fdd-99c5075979d5 None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:42.232979 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d707e70-d9b1-49b7-8fdd-99c5075979d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:42.233317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d707e70-d9b1-49b7-8fdd-99c5075979d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:42.234755 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d707e70-d9b1-49b7-8fdd-99c5075979d5 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:42.241811 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359d16a-b9b7-4e41-98f2-73e7c0d238b7 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:42.251768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 404/823] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:42.251768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d707e70-d9b1-49b7-8fdd-99c5075979d5 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:42.275924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:42.283651 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3954225-a75b-4057-9fb6-1612d9bf82a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:42.284305 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3954225-a75b-4057-9fb6-1612d9bf82a3 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:42.284582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3954225-a75b-4057-9fb6-1612d9bf82a3 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:42.284931 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3954225-a75b-4057-9fb6-1612d9bf82a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:42.369389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3954225-a75b-4057-9fb6-1612d9bf82a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:42.371208 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3954225-a75b-4057-9fb6-1612d9bf82a3 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:42.379551 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d707e70-d9b1-49b7-8fdd-99c5075979d5 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:42.383317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:42.439057 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ad626-acec-4c9e-bb91-b7bacfed19de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:42.440586 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ad626-acec-4c9e-bb91-b7bacfed19de None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:42.441064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ad626-acec-4c9e-bb91-b7bacfed19de None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:42.441555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ad626-acec-4c9e-bb91-b7bacfed19de None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:42.442791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ad626-acec-4c9e-bb91-b7bacfed19de None admin] RBAC: Adding query filter params (type=network) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:42.443297 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ad626-acec-4c9e-bb91-b7bacfed19de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:42.445968 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ad626-acec-4c9e-bb91-b7bacfed19de None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:42.458203 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 405/824] 10.222.0.71 () {56 vars in 1112 bytes} [Thu Jul 21 20:26:42 2022] GET /identity/v3/services?type=network => generated 350 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:42.460457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5b06e-32bf-4fdd-ad0a-1cd3ae484e02 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:42.461016 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5b06e-32bf-4fdd-ad0a-1cd3ae484e02 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:42.461556 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5b06e-32bf-4fdd-ad0a-1cd3ae484e02 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:42.464115 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 406/825] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:42.465323 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5b06e-32bf-4fdd-ad0a-1cd3ae484e02 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:42.537557 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 420/826] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:26:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:26:42.538365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3954225-a75b-4057-9fb6-1612d9bf82a3 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:42.606808 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:42.642979 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5b06e-32bf-4fdd-ad0a-1cd3ae484e02 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:42.691940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3954225-a75b-4057-9fb6-1612d9bf82a3 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:42.699292 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc49308-87a1-40d6-933e-6a4f13285209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:42.700205 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc49308-87a1-40d6-933e-6a4f13285209 None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:42.700496 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc49308-87a1-40d6-933e-6a4f13285209 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:42.700752 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc49308-87a1-40d6-933e-6a4f13285209 None admin] PATH_INFO: `/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/3babb078203647e2b3ef3b2787b84e3b/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:42.702920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc49308-87a1-40d6-933e-6a4f13285209 None admin] RBAC: Authorizing `identity:create_grant(project_id=37e4afa595894a9994a8e6981ebb7c41, user_id=3babb078203647e2b3ef3b2787b84e3b, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:42.704380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc49308-87a1-40d6-933e-6a4f13285209 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:42.706133 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:42.726451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 407/827] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:26:42 2022] PUT /identity/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/3babb078203647e2b3ef3b2787b84e3b/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:26:42.729023 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c8aa8-99e2-4cda-a7d1-a174776fa248 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:42.730045 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c8aa8-99e2-4cda-a7d1-a174776fa248 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:42.731418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c8aa8-99e2-4cda-a7d1-a174776fa248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:42.745809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb286c8-3ca2-4073-981a-e7b39a3d1681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:42.746750 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb286c8-3ca2-4073-981a-e7b39a3d1681 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:42.747064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb286c8-3ca2-4073-981a-e7b39a3d1681 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:42.747310 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb286c8-3ca2-4073-981a-e7b39a3d1681 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:42.748153 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb286c8-3ca2-4073-981a-e7b39a3d1681 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:42.749799 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb286c8-3ca2-4073-981a-e7b39a3d1681 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:42.762237 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-805c8aa8-99e2-4cda-a7d1-a174776fa248 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:42.764016 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-805c8aa8-99e2-4cda-a7d1-a174776fa248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:42.789782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 421/828] 10.222.0.71 () {60 vars in 1144 bytes} [Thu Jul 21 20:26:42 2022] POST /identity/v3/endpoints => generated 327 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:26:42.790607 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb286c8-3ca2-4073-981a-e7b39a3d1681 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:42.889518 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 408/829] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:26:42.929521 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c8aa8-99e2-4cda-a7d1-a174776fa248 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:43.034196 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:43.044715 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb286c8-3ca2-4073-981a-e7b39a3d1681 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:43.079229 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c8aa8-99e2-4cda-a7d1-a174776fa248 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:43.096129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39add56-e61c-4c6f-b297-7297fc99a96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:43.096993 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39add56-e61c-4c6f-b297-7297fc99a96a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:43.097647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39add56-e61c-4c6f-b297-7297fc99a96a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:43.098017 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39add56-e61c-4c6f-b297-7297fc99a96a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:43.118294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39add56-e61c-4c6f-b297-7297fc99a96a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:43.120051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39add56-e61c-4c6f-b297-7297fc99a96a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:43.133364 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96efbb6-919f-4e56-9639-6ac8e773d9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:43.134183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96efbb6-919f-4e56-9639-6ac8e773d9dd None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:43.134465 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96efbb6-919f-4e56-9639-6ac8e773d9dd None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:43.134740 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96efbb6-919f-4e56-9639-6ac8e773d9dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:43.140989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-a39add56-e61c-4c6f-b297-7297fc99a96a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:43.140989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:43.140989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:43.140989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:43.140989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:43.140989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:43.140989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:43.140989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:43.140989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:43.140989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:43.140989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:43.140989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:43.140989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:43.140989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:43.140989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:43.141986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:43.141986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:43.141986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:43.141986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:43.141986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:43.141986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:43.141986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:43.141986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:43.141986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:43.141986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:43.141986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:43.141986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:43.141986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:43.141986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:43.141986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:43.141986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:43.141986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:43.142905 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:43.142905 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:43.142905 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:43.142905 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:43.144841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 422/830] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:43.148498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96efbb6-919f-4e56-9639-6ac8e773d9dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:43.150972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39add56-e61c-4c6f-b297-7297fc99a96a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:43.164077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-c96efbb6-919f-4e56-9639-6ac8e773d9dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:43.164077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:43.164077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:43.164077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:43.164077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:43.164077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:43.164077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:43.164077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:43.164077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:43.164077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:43.164077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:43.164077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:43.164077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:43.164077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:43.164077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:43.164077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:43.165351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:43.165351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:43.165351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:43.165351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:43.165351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:43.165351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:43.165351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:43.165351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:43.165351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:43.165351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:43.165351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:43.165351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:43.165351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:43.165351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:43.165351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:43.165351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:43.166461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:43.166461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:43.166461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:43.166461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:43.166461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:43.166461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:43.166461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:43.166461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:43.166461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:43.166461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:43.166461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:26:43.166461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:43.166461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 409/831] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:26:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:26:43.171659 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96efbb6-919f-4e56-9639-6ac8e773d9dd None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:43.178112 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:43.278656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39add56-e61c-4c6f-b297-7297fc99a96a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:43.290680 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96efbb6-919f-4e56-9639-6ac8e773d9dd None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:43.319859 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cfb89-e2c4-4dc4-9aa2-9d9dc1eb8dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:43.320845 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cfb89-e2c4-4dc4-9aa2-9d9dc1eb8dc9 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:43.321330 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cfb89-e2c4-4dc4-9aa2-9d9dc1eb8dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:43.321861 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cfb89-e2c4-4dc4-9aa2-9d9dc1eb8dc9 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:43.322838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0cfb89-e2c4-4dc4-9aa2-9d9dc1eb8dc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:43.324997 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:43.325387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0cfb89-e2c4-4dc4-9aa2-9d9dc1eb8dc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:43.329923 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0cfb89-e2c4-4dc4-9aa2-9d9dc1eb8dc9 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:43.333024 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e5e47-dd29-4ada-8e3a-3ab5aa693764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:43.334048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e5e47-dd29-4ada-8e3a-3ab5aa693764 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:43.334048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e5e47-dd29-4ada-8e3a-3ab5aa693764 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:43.334279 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e5e47-dd29-4ada-8e3a-3ab5aa693764 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:43.335048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e5e47-dd29-4ada-8e3a-3ab5aa693764 None admin] RBAC: Adding query filter params (name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:43.335495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e5e47-dd29-4ada-8e3a-3ab5aa693764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:43.336920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e5e47-dd29-4ada-8e3a-3ab5aa693764 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:43.347071 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 410/832] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:26:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:43.347722 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 423/833] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:43.353737 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e5e47-dd29-4ada-8e3a-3ab5aa693764 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:43.354872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cfb89-e2c4-4dc4-9aa2-9d9dc1eb8dc9 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:43.461462 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:43.465640 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e5e47-dd29-4ada-8e3a-3ab5aa693764 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:43.478977 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cfb89-e2c4-4dc4-9aa2-9d9dc1eb8dc9 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:43.505102 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad0810-31dc-4c76-a391-949be6672022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:43.505913 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad0810-31dc-4c76-a391-949be6672022 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:43.506197 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad0810-31dc-4c76-a391-949be6672022 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:43.506471 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad0810-31dc-4c76-a391-949be6672022 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:43.518386 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad0810-31dc-4c76-a391-949be6672022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:43.519154 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df51ab-1c93-4cb8-ad5c-c5f3da4a2006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:43.519862 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad0810-31dc-4c76-a391-949be6672022 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:43.520177 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df51ab-1c93-4cb8-ad5c-c5f3da4a2006 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:43.520569 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df51ab-1c93-4cb8-ad5c-c5f3da4a2006 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:43.520948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df51ab-1c93-4cb8-ad5c-c5f3da4a2006 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:43.529970 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df51ab-1c93-4cb8-ad5c-c5f3da4a2006 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:43.531391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df51ab-1c93-4cb8-ad5c-c5f3da4a2006 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:43.539748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-02ad0810-31dc-4c76-a391-949be6672022 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:43.539748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:43.539748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:43.539748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:43.539748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:43.539748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:43.539748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:43.539748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:43.539748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:43.539748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:43.539748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:43.539748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:43.539748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:43.539748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:43.539748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:43.540918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:43.540918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:43.540918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:43.540918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:43.540918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:43.540918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:43.540918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:43.540918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:43.540918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:43.540918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:43.540918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:43.540918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:43.540918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:43.540918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:43.540918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:43.540918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:43.540918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:43.542075 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:43.542075 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:43.542075 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:43.542075 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:43.549112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 411/834] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:43.550248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad0810-31dc-4c76-a391-949be6672022 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:43.551480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-93df51ab-1c93-4cb8-ad5c-c5f3da4a2006 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:43.551480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:43.551480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:43.551480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:43.551480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:43.551480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:43.551480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:43.551480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:43.551480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:43.551480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:43.551480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:43.551480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:43.551480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:43.551480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:43.551480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:43.552532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:43.552532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:43.552532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:43.552532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:43.552532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:43.552532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:43.552532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:43.552532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:43.552532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:43.552532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:43.552532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:43.552532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:43.552532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:43.552532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:43.552532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:43.552532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:43.552532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:43.553683 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:43.553683 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:43.553683 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:43.553683 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:43.553683 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:43.553683 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:43.553683 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:43.553683 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:43.560325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 424/835] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:43.561588 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df51ab-1c93-4cb8-ad5c-c5f3da4a2006 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:43.606221 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:43.666314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad0810-31dc-4c76-a391-949be6672022 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:43.686340 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df51ab-1c93-4cb8-ad5c-c5f3da4a2006 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:43.705343 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e260af-f6d5-4c2a-8d0d-61e12aa1054f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:43.706343 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e260af-f6d5-4c2a-8d0d-61e12aa1054f None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:43.706794 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e260af-f6d5-4c2a-8d0d-61e12aa1054f None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:43.707225 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e260af-f6d5-4c2a-8d0d-61e12aa1054f None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:43.708144 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e260af-f6d5-4c2a-8d0d-61e12aa1054f None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:43.708623 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e260af-f6d5-4c2a-8d0d-61e12aa1054f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:43.710265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e260af-f6d5-4c2a-8d0d-61e12aa1054f None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:43.728411 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3e273-f8e1-49b1-84fe-b466d817f1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:43.728994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3e273-f8e1-49b1-84fe-b466d817f1c1 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:43.729234 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 412/836] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:43.729906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3e273-f8e1-49b1-84fe-b466d817f1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:43.730812 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3e273-f8e1-49b1-84fe-b466d817f1c1 None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:43.731632 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e260af-f6d5-4c2a-8d0d-61e12aa1054f None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:43.731835 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d3e273-f8e1-49b1-84fe-b466d817f1c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:43.732251 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d3e273-f8e1-49b1-84fe-b466d817f1c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:43.733921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d3e273-f8e1-49b1-84fe-b466d817f1c1 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:43.742787 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:43.755921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 425/837] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:43.758171 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3e273-f8e1-49b1-84fe-b466d817f1c1 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:43.861415 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e260af-f6d5-4c2a-8d0d-61e12aa1054f None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:43.864685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:43.875875 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3e273-f8e1-49b1-84fe-b466d817f1c1 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:43.907949 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae590a-c6b0-4bfa-8f83-6930067a031b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:43.908757 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae590a-c6b0-4bfa-8f83-6930067a031b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:43.909035 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae590a-c6b0-4bfa-8f83-6930067a031b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:43.909341 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae590a-c6b0-4bfa-8f83-6930067a031b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:43.915651 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc7cc6-df68-4476-959a-11be308bd03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:43.916457 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc7cc6-df68-4476-959a-11be308bd03e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:43.916741 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc7cc6-df68-4476-959a-11be308bd03e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:43.916918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae590a-c6b0-4bfa-8f83-6930067a031b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:43.917019 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc7cc6-df68-4476-959a-11be308bd03e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:43.917913 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc7cc6-df68-4476-959a-11be308bd03e None admin] RBAC: Adding query filter params (role.id=19f5e72c0be44a1197c7288de82e79e8, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=6fbc5240f5214aaea4418052b94739e2) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:43.918232 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc7cc6-df68-4476-959a-11be308bd03e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:43.918548 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae590a-c6b0-4bfa-8f83-6930067a031b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:43.919662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc7cc6-df68-4476-959a-11be308bd03e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:43.933268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 426/838] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:26:43 2022] GET /identity/v3/role_assignments?user.id=6fbc5240f5214aaea4418052b94739e2&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:43.933268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f852b-7db7-4882-ae5c-e873bc98539a None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:43.933268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f852b-7db7-4882-ae5c-e873bc98539a None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:43.933268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f852b-7db7-4882-ae5c-e873bc98539a None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:43.933268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 427/839] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:43.935416 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a464a-1b1e-4c2f-9129-aa39fb0dbee8 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:43.937414 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a464a-1b1e-4c2f-9129-aa39fb0dbee8 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:43.937414 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a464a-1b1e-4c2f-9129-aa39fb0dbee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:43.944486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-e6ae590a-c6b0-4bfa-8f83-6930067a031b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:26:43.944486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:43.944486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:43.944486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:43.944486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:43.944486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:43.944486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:43.944486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:43.944486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:43.944486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:43.944486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:43.944486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:43.944486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:43.944486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:43.944486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:43.944486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:43.945739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:43.945739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:43.945739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:43.945739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:43.945739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:43.945739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:43.945739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:43.945739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:43.945739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:43.945739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:43.945739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:43.945739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:43.945739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:43.945739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:43.945739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:43.945739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:43.945739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:43.945739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:43.946923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:43.946923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:43.946923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:43.946923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:43.946923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:43.946923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:43.946923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:43.946923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:43.946923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:26:43.946923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:43.954147 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 413/840] 10.222.0.71 () {56 vars in 1117 bytes} [Thu Jul 21 20:26:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:26:43.955229 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae590a-c6b0-4bfa-8f83-6930067a031b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:43.955846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-f57a464a-1b1e-4c2f-9129-aa39fb0dbee8 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:43.957269 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f57a464a-1b1e-4c2f-9129-aa39fb0dbee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:44.025638 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:44.041229 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 428/841] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:26:44.065810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a464a-1b1e-4c2f-9129-aa39fb0dbee8 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:44.104341 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae590a-c6b0-4bfa-8f83-6930067a031b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:44.148027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f615f4-8c66-4194-b430-f98307a203dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:44.148776 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f615f4-8c66-4194-b430-f98307a203dd None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:44.149062 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f615f4-8c66-4194-b430-f98307a203dd None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:44.149364 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f615f4-8c66-4194-b430-f98307a203dd None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:44.150732 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f615f4-8c66-4194-b430-f98307a203dd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:44.151054 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f615f4-8c66-4194-b430-f98307a203dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:44.153315 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f615f4-8c66-4194-b430-f98307a203dd None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:44.176312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 414/842] 10.222.0.71 () {56 vars in 1149 bytes} [Thu Jul 21 20:26:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:44.182554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f615f4-8c66-4194-b430-f98307a203dd None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:44.188482 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a464a-1b1e-4c2f-9129-aa39fb0dbee8 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:44.221478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:44.236332 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59baa0b6-5c44-4e22-8e32-a768f53f8db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:44.237721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baa0b6-5c44-4e22-8e32-a768f53f8db7 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:44.238608 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baa0b6-5c44-4e22-8e32-a768f53f8db7 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:44.239399 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baa0b6-5c44-4e22-8e32-a768f53f8db7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:44.240290 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59baa0b6-5c44-4e22-8e32-a768f53f8db7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:44.242340 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59baa0b6-5c44-4e22-8e32-a768f53f8db7 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:44.257930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-59baa0b6-5c44-4e22-8e32-a768f53f8db7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 20:26:44.257930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:44.257930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:26:44.257930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:26:44.257930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:26:44.257930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:26:44.257930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:26:44.257930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:44.257930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 20:26:44.257930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:44.257930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:44.257930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:44.257930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:44.257930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:44.257930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:44.257930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:44.257930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:44.258961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:44.258961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:44.258961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:44.258961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:44.258961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:44.258961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:44.258961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:26:44.258961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:26:44.258961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:26:44.258961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:26:44.258961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:44.258961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:44.258961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:44.258961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:44.258961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:44.258961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:44.259802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:44.259802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:44.259802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:44.259802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:44.259802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:44.259802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:44.259802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:44.259802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:44.259802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:44.259802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:44.259802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:26:44.259802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:44.259802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 20:26:44.259802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:44.262814 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 429/843] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:26:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:26:44.271996 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59baa0b6-5c44-4e22-8e32-a768f53f8db7 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:44.301710 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f615f4-8c66-4194-b430-f98307a203dd None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:44.344510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:44.346439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8087e86-fcfa-4949-a491-8f657dd94ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:44.347400 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8087e86-fcfa-4949-a491-8f657dd94ae5 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:44.347889 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8087e86-fcfa-4949-a491-8f657dd94ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:44.348377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8087e86-fcfa-4949-a491-8f657dd94ae5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:44.394738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59baa0b6-5c44-4e22-8e32-a768f53f8db7 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:44.431361 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad3047-6050-4a66-b780-82740a3c0d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:44.432373 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad3047-6050-4a66-b780-82740a3c0d0e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:44.432870 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad3047-6050-4a66-b780-82740a3c0d0e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:44.433547 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad3047-6050-4a66-b780-82740a3c0d0e None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:44.434526 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad3047-6050-4a66-b780-82740a3c0d0e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:44.435036 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad3047-6050-4a66-b780-82740a3c0d0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:44.436614 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad3047-6050-4a66-b780-82740a3c0d0e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:44.439854 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8087e86-fcfa-4949-a491-8f657dd94ae5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:44.441623 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8087e86-fcfa-4949-a491-8f657dd94ae5 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:44.447115 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 430/844] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:26:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:44.448278 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad3047-6050-4a66-b780-82740a3c0d0e None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:44.481963 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:44.568568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 415/845] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:26:44 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 20:26:44.575439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8087e86-fcfa-4949-a491-8f657dd94ae5 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:44.577788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad3047-6050-4a66-b780-82740a3c0d0e None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:44.599499 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:44.616210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5349b65-dc31-479d-8d4f-62a57608e049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:44.616878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5349b65-dc31-479d-8d4f-62a57608e049 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:44.617389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5349b65-dc31-479d-8d4f-62a57608e049 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:44.617500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5349b65-dc31-479d-8d4f-62a57608e049 None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:44.618171 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5349b65-dc31-479d-8d4f-62a57608e049 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:44.618479 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5349b65-dc31-479d-8d4f-62a57608e049 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:44.619852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5349b65-dc31-479d-8d4f-62a57608e049 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:44.625436 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 431/846] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:26:44 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:44.698877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8087e86-fcfa-4949-a491-8f657dd94ae5 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:44.725636 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:44.738791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef6081-48c7-4a79-b70d-e53bfea621af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:44.739862 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef6081-48c7-4a79-b70d-e53bfea621af None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:44.740290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef6081-48c7-4a79-b70d-e53bfea621af None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:44.740705 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef6081-48c7-4a79-b70d-e53bfea621af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:44.753356 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef6081-48c7-4a79-b70d-e53bfea621af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:44.755146 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef6081-48c7-4a79-b70d-e53bfea621af None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:44.774734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-60ef6081-48c7-4a79-b70d-e53bfea621af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:44.774734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:44.774734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:44.774734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:44.774734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:44.774734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:44.774734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:44.774734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:44.774734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:44.774734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:44.774734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:44.774734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:44.774734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:44.774734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:44.774734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:44.775873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:44.775873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:44.775873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:44.775873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:44.775873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:44.775873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:44.775873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:44.775873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:44.775873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:44.775873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:44.775873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:44.775873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:44.775873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:44.775873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:44.775873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:44.775873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:44.775873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:44.776988 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:44.776988 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:44.776988 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:44.776988 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:44.779518 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 416/847] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:26:44.785193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5349b65-dc31-479d-8d4f-62a57608e049 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:44.894148 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5349b65-dc31-479d-8d4f-62a57608e049 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:44.929712 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2c429-46d6-4542-b3f7-5b5973c54a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:44.930377 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2c429-46d6-4542-b3f7-5b5973c54a43 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:44.930657 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2c429-46d6-4542-b3f7-5b5973c54a43 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:44.930930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2c429-46d6-4542-b3f7-5b5973c54a43 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:44.931663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2c429-46d6-4542-b3f7-5b5973c54a43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:44.931968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2c429-46d6-4542-b3f7-5b5973c54a43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:44.933407 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2c429-46d6-4542-b3f7-5b5973c54a43 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:44.946366 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 432/848] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:44.951817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef6081-48c7-4a79-b70d-e53bfea621af None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:45.060402 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef6081-48c7-4a79-b70d-e53bfea621af None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:45.083728 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb59c26-eeb3-4a5d-a5c7-37f652f4c7a8 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:45.084020 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb59c26-eeb3-4a5d-a5c7-37f652f4c7a8 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:45.084305 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb59c26-eeb3-4a5d-a5c7-37f652f4c7a8 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:45.086999 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 433/849] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:26:45.092085 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d5318-1f3e-46e5-b940-62b5973ce9a4 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:45.092372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d5318-1f3e-46e5-b940-62b5973ce9a4 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:45.092655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d5318-1f3e-46e5-b940-62b5973ce9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:45.098236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4f648-8c28-420b-8229-bff5583a6d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:45.099653 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4f648-8c28-420b-8229-bff5583a6d55 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:45.099998 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4f648-8c28-420b-8229-bff5583a6d55 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:45.100271 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4f648-8c28-420b-8229-bff5583a6d55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:45.107651 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4f648-8c28-420b-8229-bff5583a6d55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:45.109177 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4f648-8c28-420b-8229-bff5583a6d55 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:45.112069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-de2d5318-1f3e-46e5-b940-62b5973ce9a4 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:45.113663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-de2d5318-1f3e-46e5-b940-62b5973ce9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:45.124060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-1fd4f648-8c28-420b-8229-bff5583a6d55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:45.124060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:45.124060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:45.124060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:45.124060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:45.124060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:45.124060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:45.124060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:45.124060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:45.124060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:45.124060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:45.124060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:45.124060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:45.124060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:45.124060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:45.125025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:45.125025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:45.125025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:45.125025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:45.125025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:45.125025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:45.125025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:45.125025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:45.125025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:45.125025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:45.125025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:45.125025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:45.125025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:45.125025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:45.125025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:45.125025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:45.125025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:45.126653 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:45.126653 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:45.126653 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:45.126653 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:45.126653 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:45.126653 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:45.126653 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:45.126653 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:45.126653 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 417/850] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:45.134086 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4f648-8c28-420b-8229-bff5583a6d55 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:45.165866 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:45.185602 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:45.193906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 434/851] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:26:45.222250 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d5318-1f3e-46e5-b940-62b5973ce9a4 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:45.256964 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4f648-8c28-420b-8229-bff5583a6d55 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:45.299516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022974f6-eb9f-4e3c-a38b-e29ac1c5c003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:45.300209 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022974f6-eb9f-4e3c-a38b-e29ac1c5c003 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:45.300488 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022974f6-eb9f-4e3c-a38b-e29ac1c5c003 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:45.300757 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022974f6-eb9f-4e3c-a38b-e29ac1c5c003 None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:45.301575 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022974f6-eb9f-4e3c-a38b-e29ac1c5c003 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:45.301881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022974f6-eb9f-4e3c-a38b-e29ac1c5c003 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:45.303844 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022974f6-eb9f-4e3c-a38b-e29ac1c5c003 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:45.320859 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 418/852] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:45.330380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022974f6-eb9f-4e3c-a38b-e29ac1c5c003 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:45.336467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d5318-1f3e-46e5-b940-62b5973ce9a4 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:45.381853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f30ba-cc87-46e8-afcf-54d77ecd9769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:45.383770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f30ba-cc87-46e8-afcf-54d77ecd9769 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:45.384324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f30ba-cc87-46e8-afcf-54d77ecd9769 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:45.385086 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f30ba-cc87-46e8-afcf-54d77ecd9769 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:45.401189 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f30ba-cc87-46e8-afcf-54d77ecd9769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:45.412282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-3c2f30ba-cc87-46e8-afcf-54d77ecd9769 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:26:45.412282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:45.412282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:45.412282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:45.412282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:45.412282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:45.412282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:45.412282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:45.412282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:45.412282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:45.412282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:45.412282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:45.412282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:45.412282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:45.412282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:45.412282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:45.413318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:45.413318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:45.413318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:45.413318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:45.413318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:45.413318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:45.413318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:45.413318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:45.413318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:45.413318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:45.413318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:45.413318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:45.413318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:45.413318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:45.413318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:45.413318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:45.414179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:45.414179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:45.414179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:45.414179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:45.414179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:45.414179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:45.414179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:45.414179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:45.414179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:45.414179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:45.414179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:26:45.414179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:45.423093 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 435/853] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:26:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:45.424941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f30ba-cc87-46e8-afcf-54d77ecd9769 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:45.441579 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:45.455599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022974f6-eb9f-4e3c-a38b-e29ac1c5c003 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:45.470595 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:45.503744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362d1a2-7122-401e-a1dd-7043cc0e90cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:45.504944 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362d1a2-7122-401e-a1dd-7043cc0e90cb None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:45.505436 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362d1a2-7122-401e-a1dd-7043cc0e90cb None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:45.506150 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362d1a2-7122-401e-a1dd-7043cc0e90cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:45.507652 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362d1a2-7122-401e-a1dd-7043cc0e90cb None admin] RBAC: Adding query filter params (role.id=9a48ab1bf32a45828e9e73599c980dac, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=b94e634e8855486a9ce14d3de9d986fb) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:45.508160 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362d1a2-7122-401e-a1dd-7043cc0e90cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:45.510353 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362d1a2-7122-401e-a1dd-7043cc0e90cb None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:45.519721 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 419/854] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:26:45 2022] GET /identity/v3/role_assignments?user.id=b94e634e8855486a9ce14d3de9d986fb&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=9a48ab1bf32a45828e9e73599c980dac => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:45.556484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f30ba-cc87-46e8-afcf-54d77ecd9769 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:45.597398 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca93b7-dfed-4319-a242-d720268ec144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:45.598405 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca93b7-dfed-4319-a242-d720268ec144 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:45.599126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca93b7-dfed-4319-a242-d720268ec144 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:45.599620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca93b7-dfed-4319-a242-d720268ec144 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:45.600581 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca93b7-dfed-4319-a242-d720268ec144 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:45.601120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca93b7-dfed-4319-a242-d720268ec144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:45.602732 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca93b7-dfed-4319-a242-d720268ec144 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:45.617199 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 436/855] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:26:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:45.619711 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362d1a2-7122-401e-a1dd-7043cc0e90cb None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:45.729917 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362d1a2-7122-401e-a1dd-7043cc0e90cb None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:45.774783 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a14ee9-bc5d-4f94-858f-c6b7079c8ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:45.775540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a14ee9-bc5d-4f94-858f-c6b7079c8ca3 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:45.775829 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a14ee9-bc5d-4f94-858f-c6b7079c8ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:45.776096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a14ee9-bc5d-4f94-858f-c6b7079c8ca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:45.791861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a14ee9-bc5d-4f94-858f-c6b7079c8ca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:45.795211 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a14ee9-bc5d-4f94-858f-c6b7079c8ca3 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:45.817264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-e9a14ee9-bc5d-4f94-858f-c6b7079c8ca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:45.817264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:45.817264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:45.817264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:45.817264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:45.817264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:45.817264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:45.817264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:45.817264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:45.817264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:45.817264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:45.817264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:45.817264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:45.817264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:45.817264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:45.818306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:45.818306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:45.818306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:45.818306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:45.818306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:45.818306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:45.818306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:45.818306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:45.818306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:45.818306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:45.818306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:45.818306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:45.818306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:45.818306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:45.818306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:45.818306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:45.818306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:45.819562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:45.819562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:45.819562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:45.819562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:45.819562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 420/856] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:45.826226 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca93b7-dfed-4319-a242-d720268ec144 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:46.050289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:46.057716 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca93b7-dfed-4319-a242-d720268ec144 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:46.094596 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1ec2c-3ac2-46c4-89fc-36bcb70e4b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:46.095598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1ec2c-3ac2-46c4-89fc-36bcb70e4b7a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:46.096091 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1ec2c-3ac2-46c4-89fc-36bcb70e4b7a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:46.096559 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1ec2c-3ac2-46c4-89fc-36bcb70e4b7a None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:46.097521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1ec2c-3ac2-46c4-89fc-36bcb70e4b7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:46.098030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1ec2c-3ac2-46c4-89fc-36bcb70e4b7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:46.099855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1ec2c-3ac2-46c4-89fc-36bcb70e4b7a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:46.112652 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 437/857] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:46.118656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a14ee9-bc5d-4f94-858f-c6b7079c8ca3 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:46.132757 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:46.198087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f673e-0698-4260-bdfe-cf42a442b346 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:46.198763 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f673e-0698-4260-bdfe-cf42a442b346 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:46.199855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f673e-0698-4260-bdfe-cf42a442b346 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:46.201939 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 438/858] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:26:46.208799 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c2828-bdac-4ef0-b2dc-e8bbae3389bf None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:46.208799 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c2828-bdac-4ef0-b2dc-e8bbae3389bf None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:46.208799 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c2828-bdac-4ef0-b2dc-e8bbae3389bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:46.226250 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-e48c2828-bdac-4ef0-b2dc-e8bbae3389bf None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:46.227811 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-e48c2828-bdac-4ef0-b2dc-e8bbae3389bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:46.229526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a14ee9-bc5d-4f94-858f-c6b7079c8ca3 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:46.267359 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7983d-b810-487b-8bbc-91d0ca11f889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:46.268165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7983d-b810-487b-8bbc-91d0ca11f889 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:46.268448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7983d-b810-487b-8bbc-91d0ca11f889 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:46.268722 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7983d-b810-487b-8bbc-91d0ca11f889 None admin] PATH_INFO: `/v3/users/swift` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:46.276885 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7983d-b810-487b-8bbc-91d0ca11f889 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:46.278416 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7983d-b810-487b-8bbc-91d0ca11f889 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:46.294346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-dbe7983d-b810-487b-8bbc-91d0ca11f889 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:26:46.294346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:46.294346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:46.294346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:46.294346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:46.294346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:46.294346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:46.294346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:46.294346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:46.294346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:46.294346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:46.294346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:46.294346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:46.294346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:46.294346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:46.294346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:46.295375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:46.295375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:46.295375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:46.295375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:46.295375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:46.295375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:46.295375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:46.295375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:46.295375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:46.295375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:46.295375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:46.295375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:46.295375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:46.295375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:46.295375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:46.295375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:46.295375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:46.295375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:46.296339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:46.296339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:46.296339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:46.296339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:46.296339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:46.296339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:46.296339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:46.296339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:46.296339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:26:46.296339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:46.296339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 421/859] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:26:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:46.302469 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7983d-b810-487b-8bbc-91d0ca11f889 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:46.325093 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 439/860] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:26:46.341271 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c2828-bdac-4ef0-b2dc-e8bbae3389bf None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:46.399990 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:46.435876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:46.443271 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7983d-b810-487b-8bbc-91d0ca11f889 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:46.453327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c2828-bdac-4ef0-b2dc-e8bbae3389bf None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:46.488208 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854262f-408f-439d-a07b-cffdf6e20e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:46.488882 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854262f-408f-439d-a07b-cffdf6e20e73 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:46.489212 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854262f-408f-439d-a07b-cffdf6e20e73 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:46.489486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854262f-408f-439d-a07b-cffdf6e20e73 None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:46.490328 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854262f-408f-439d-a07b-cffdf6e20e73 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:46.490634 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854262f-408f-439d-a07b-cffdf6e20e73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:46.492045 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854262f-408f-439d-a07b-cffdf6e20e73 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:46.506217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecada77b-c334-464c-a837-02b0120630f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:46.507912 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecada77b-c334-464c-a837-02b0120630f2 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:46.508407 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecada77b-c334-464c-a837-02b0120630f2 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:46.509182 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecada77b-c334-464c-a837-02b0120630f2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:46.510076 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecada77b-c334-464c-a837-02b0120630f2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:46.511167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 422/861] 10.222.0.71 () {56 vars in 1135 bytes} [Thu Jul 21 20:26:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:46.511947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecada77b-c334-464c-a837-02b0120630f2 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:46.516602 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854262f-408f-439d-a07b-cffdf6e20e73 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:46.528978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-ecada77b-c334-464c-a837-02b0120630f2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 20:26:46.528978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:46.528978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:26:46.528978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:26:46.528978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:26:46.528978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:26:46.528978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:26:46.528978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:46.528978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 20:26:46.528978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:46.528978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:46.528978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:46.528978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:46.528978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:46.528978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:46.528978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:46.530105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:46.530105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:46.530105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:46.530105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:46.530105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:46.530105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:46.530105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:46.530105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:26:46.530105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:26:46.530105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:26:46.530105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:26:46.530105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:46.530105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:46.530105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:46.530105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:46.530105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:46.530955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:46.530955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:46.530955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:46.530955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:46.530955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:46.530955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:46.530955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:46.530955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:46.530955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:46.530955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:46.530955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:46.530955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:26:46.530955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:46.530955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 20:26:46.530955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:46.534387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 440/862] 10.222.0.71 () {56 vars in 1132 bytes} [Thu Jul 21 20:26:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:26:46.539791 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecada77b-c334-464c-a837-02b0120630f2 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:46.633634 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854262f-408f-439d-a07b-cffdf6e20e73 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:46.664695 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:46.675255 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743dd3c3-d83c-4dc6-92c3-966cb4a17341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:46.675910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dd3c3-d83c-4dc6-92c3-966cb4a17341 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:46.676194 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dd3c3-d83c-4dc6-92c3-966cb4a17341 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:46.676703 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dd3c3-d83c-4dc6-92c3-966cb4a17341 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:46.683723 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecada77b-c334-464c-a837-02b0120630f2 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:46.723108 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3905f7a-fad2-4528-a982-8474dd3979ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:46.723778 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3905f7a-fad2-4528-a982-8474dd3979ea None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:46.724319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3905f7a-fad2-4528-a982-8474dd3979ea None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:46.724655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3905f7a-fad2-4528-a982-8474dd3979ea None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:46.725474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3905f7a-fad2-4528-a982-8474dd3979ea None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:46.725785 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3905f7a-fad2-4528-a982-8474dd3979ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:46.727161 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3905f7a-fad2-4528-a982-8474dd3979ea None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:46.737332 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 441/863] 10.222.0.71 () {56 vars in 1126 bytes} [Thu Jul 21 20:26:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:46.738927 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3905f7a-fad2-4528-a982-8474dd3979ea None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:46.755631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:46.764847 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743dd3c3-d83c-4dc6-92c3-966cb4a17341 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:46.766368 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743dd3c3-d83c-4dc6-92c3-966cb4a17341 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:46.851219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3905f7a-fad2-4528-a982-8474dd3979ea None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:46.879498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 423/864] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:26:46 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:26:46.880832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3e36d-b579-447f-b1cf-50f591c3e117 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:46.881150 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3e36d-b579-447f-b1cf-50f591c3e117 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:46.881426 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3e36d-b579-447f-b1cf-50f591c3e117 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:46.882847 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 424/865] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:46.885959 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3e36d-b579-447f-b1cf-50f591c3e117 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:46.894044 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb30109-c469-44a1-8a37-5edc9d81a5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:46.894713 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb30109-c469-44a1-8a37-5edc9d81a5ce None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:46.894996 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb30109-c469-44a1-8a37-5edc9d81a5ce None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:46.895272 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb30109-c469-44a1-8a37-5edc9d81a5ce None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:46.895988 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb30109-c469-44a1-8a37-5edc9d81a5ce None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:46.896296 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb30109-c469-44a1-8a37-5edc9d81a5ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:46.897708 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb30109-c469-44a1-8a37-5edc9d81a5ce None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:46.913119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 442/866] 10.222.0.71 () {56 vars in 1126 bytes} [Thu Jul 21 20:26:46 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:46.913119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f995333-d4b9-4322-ab8b-65fc781d97f9 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:46.913119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f995333-d4b9-4322-ab8b-65fc781d97f9 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:46.913119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f995333-d4b9-4322-ab8b-65fc781d97f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:46.922593 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-3f995333-d4b9-4322-ab8b-65fc781d97f9 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:46.923815 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3f995333-d4b9-4322-ab8b-65fc781d97f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:46.932128 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:47.017119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 443/867] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:26:47.038047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f995333-d4b9-4322-ab8b-65fc781d97f9 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:47.042537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3e36d-b579-447f-b1cf-50f591c3e117 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:47.050852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:47.079899 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d66376-7524-426c-aa32-12855c711ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:47.080823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d66376-7524-426c-aa32-12855c711ea4 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:47.081290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d66376-7524-426c-aa32-12855c711ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:47.082372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d66376-7524-426c-aa32-12855c711ea4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:47.094909 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d66376-7524-426c-aa32-12855c711ea4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:47.096454 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d66376-7524-426c-aa32-12855c711ea4 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:47.116079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-28d66376-7524-426c-aa32-12855c711ea4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:47.116079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:47.116079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:47.116079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:47.116079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:47.116079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:47.116079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:47.116079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:47.116079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:47.116079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:47.116079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:47.116079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:47.116079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:47.116079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:47.116079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:47.117022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:47.117022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:47.117022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:47.117022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:47.117022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:47.117022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:47.117022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:47.117022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:47.117022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:47.117022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:47.117022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:47.117022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:47.117022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:47.117022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:47.117022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:47.117022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:47.117022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:47.118032 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:47.118032 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:47.118032 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:47.118032 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:47.118032 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 425/868] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:47.123297 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d66376-7524-426c-aa32-12855c711ea4 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:47.167481 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f995333-d4b9-4322-ab8b-65fc781d97f9 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:47.205342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4c341-2afe-49f6-960e-188883faf243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:47.206178 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4c341-2afe-49f6-960e-188883faf243 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:47.206560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4c341-2afe-49f6-960e-188883faf243 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:47.206934 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4c341-2afe-49f6-960e-188883faf243 None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:47.207782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4c341-2afe-49f6-960e-188883faf243 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:47.209942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4c341-2afe-49f6-960e-188883faf243 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:47.221902 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:47.227928 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 444/869] 10.222.0.71 () {60 vars in 1141 bytes} [Thu Jul 21 20:26:47 2022] POST /identity/v3/services => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:26:47.244851 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d66376-7524-426c-aa32-12855c711ea4 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:47.294431 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34635c4-811e-4a7f-858d-22a6a01fa23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:47.295114 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34635c4-811e-4a7f-858d-22a6a01fa23f None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:47.295389 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34635c4-811e-4a7f-858d-22a6a01fa23f None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:47.295662 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34635c4-811e-4a7f-858d-22a6a01fa23f None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:47.296405 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34635c4-811e-4a7f-858d-22a6a01fa23f None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:47.296721 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34635c4-811e-4a7f-858d-22a6a01fa23f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:47.298178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34635c4-811e-4a7f-858d-22a6a01fa23f None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:47.312250 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 426/870] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:47.317849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4c341-2afe-49f6-960e-188883faf243 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:47.393233 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:47.430524 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4c341-2afe-49f6-960e-188883faf243 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:47.476434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af5aa6-857a-4997-8010-b48075c1f59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:47.477294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af5aa6-857a-4997-8010-b48075c1f59a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:47.477579 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af5aa6-857a-4997-8010-b48075c1f59a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:47.477855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af5aa6-857a-4997-8010-b48075c1f59a None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:47.487567 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af5aa6-857a-4997-8010-b48075c1f59a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:47.488765 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af5aa6-857a-4997-8010-b48075c1f59a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:47.510324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-b2af5aa6-857a-4997-8010-b48075c1f59a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:47.510324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:47.510324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:47.510324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:47.510324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:47.510324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:47.510324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:47.510324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:47.510324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:47.510324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:47.510324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:47.510324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:47.510324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:47.510324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:47.510324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:47.511317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:47.511317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:47.511317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:47.511317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:47.511317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:47.511317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:47.511317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:47.511317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:47.511317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:47.511317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:47.511317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:47.511317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:47.511317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:47.511317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:47.511317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:47.511317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:47.511317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:47.512262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:47.512262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:47.512262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:47.512262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:47.512262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:47.512262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:47.512262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:47.512262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:47.517868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 445/871] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:47.521216 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34635c4-811e-4a7f-858d-22a6a01fa23f None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:47.628468 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34635c4-811e-4a7f-858d-22a6a01fa23f None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:47.664993 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279897d6-3e72-4e45-b2cd-c704fc0328c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:47.666071 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279897d6-3e72-4e45-b2cd-c704fc0328c8 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:47.666564 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279897d6-3e72-4e45-b2cd-c704fc0328c8 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:47.667069 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279897d6-3e72-4e45-b2cd-c704fc0328c8 None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:47.668664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279897d6-3e72-4e45-b2cd-c704fc0328c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:47.669226 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279897d6-3e72-4e45-b2cd-c704fc0328c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:47.670879 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279897d6-3e72-4e45-b2cd-c704fc0328c8 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:47.674089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:47.690606 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 427/872] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:47.693654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af5aa6-857a-4997-8010-b48075c1f59a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:47.779373 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46d5d3-d67c-46af-860a-0578a73955ae None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:47.779721 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46d5d3-d67c-46af-860a-0578a73955ae None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:47.779937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46d5d3-d67c-46af-860a-0578a73955ae None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:47.781593 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 428/873] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:26:47.786688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816ecb1-39af-477a-ad13-4faef7555d30 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:47.786928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816ecb1-39af-477a-ad13-4faef7555d30 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:47.787189 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816ecb1-39af-477a-ad13-4faef7555d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:47.806407 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-e816ecb1-39af-477a-ad13-4faef7555d30 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:47.806918 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af5aa6-857a-4997-8010-b48075c1f59a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:47.807621 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-e816ecb1-39af-477a-ad13-4faef7555d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:47.837579 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:47.847723 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f9ba8-067e-43e4-aaa4-4b3b98d367db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:47.848618 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f9ba8-067e-43e4-aaa4-4b3b98d367db None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:47.849303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f9ba8-067e-43e4-aaa4-4b3b98d367db None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:47.849682 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f9ba8-067e-43e4-aaa4-4b3b98d367db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:47.850627 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f9ba8-067e-43e4-aaa4-4b3b98d367db None admin] RBAC: Adding query filter params (role.id=19f5e72c0be44a1197c7288de82e79e8, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=3babb078203647e2b3ef3b2787b84e3b) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:47.851043 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f9ba8-067e-43e4-aaa4-4b3b98d367db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:47.852604 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f9ba8-067e-43e4-aaa4-4b3b98d367db None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:47.865135 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 446/874] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:26:47 2022] GET /identity/v3/role_assignments?user.id=3babb078203647e2b3ef3b2787b84e3b&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:47.899097 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 429/875] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:47 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:26:47.924934 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f9ba8-067e-43e4-aaa4-4b3b98d367db None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:48.039543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f9ba8-067e-43e4-aaa4-4b3b98d367db None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:48.076484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ca42c-e1b5-4b96-8159-4932c65b5eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:48.077318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ca42c-e1b5-4b96-8159-4932c65b5eb2 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:48.077595 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ca42c-e1b5-4b96-8159-4932c65b5eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:48.077911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ca42c-e1b5-4b96-8159-4932c65b5eb2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:48.088636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ca42c-e1b5-4b96-8159-4932c65b5eb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:48.097553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-669ca42c-e1b5-4b96-8159-4932c65b5eb2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:26:48.097553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:48.097553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:48.097553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:48.097553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:48.097553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:48.097553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:48.097553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:48.097553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:48.097553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:48.097553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:48.097553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:48.097553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:48.097553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:48.097553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:48.097553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:48.098562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:48.098562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:48.098562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:48.098562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:48.098562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:48.098562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:48.098562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:48.098562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:48.098562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:48.098562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:48.098562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:48.098562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:48.098562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:48.098562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:48.098562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:48.098562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:48.099852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:48.099852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:48.099852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:48.099852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:48.099852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:48.099852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:48.099852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:48.099852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:48.099852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:48.099852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:48.099852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:26:48.099852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:48.099852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 447/876] 10.222.0.71 () {56 vars in 1120 bytes} [Thu Jul 21 20:26:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:48.107662 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816ecb1-39af-477a-ad13-4faef7555d30 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:48.210961 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816ecb1-39af-477a-ad13-4faef7555d30 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:48.241439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:48.246667 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce551b-a1e4-4eae-a5da-7b2a6de72f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:48.247413 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce551b-a1e4-4eae-a5da-7b2a6de72f17 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:48.247910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce551b-a1e4-4eae-a5da-7b2a6de72f17 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:48.248371 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce551b-a1e4-4eae-a5da-7b2a6de72f17 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:48.249206 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce551b-a1e4-4eae-a5da-7b2a6de72f17 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:48.249718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce551b-a1e4-4eae-a5da-7b2a6de72f17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:48.251094 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce551b-a1e4-4eae-a5da-7b2a6de72f17 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:48.260084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 430/877] 10.222.0.71 () {56 vars in 1115 bytes} [Thu Jul 21 20:26:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:48.265844 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ca42c-e1b5-4b96-8159-4932c65b5eb2 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:48.323309 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:48.379657 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ca42c-e1b5-4b96-8159-4932c65b5eb2 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:48.414219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb209e7-2088-41ed-8ca4-30b4ba4cbd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:48.415010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb209e7-2088-41ed-8ca4-30b4ba4cbd92 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:48.415291 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb209e7-2088-41ed-8ca4-30b4ba4cbd92 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:48.415566 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb209e7-2088-41ed-8ca4-30b4ba4cbd92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:48.427004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb209e7-2088-41ed-8ca4-30b4ba4cbd92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:48.428458 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb209e7-2088-41ed-8ca4-30b4ba4cbd92 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:48.445714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-9eb209e7-2088-41ed-8ca4-30b4ba4cbd92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:48.445714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:48.445714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:48.445714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:48.445714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:48.445714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:48.445714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:48.445714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:48.445714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:48.445714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:48.445714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:48.445714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:48.445714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:48.445714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:48.445714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:48.446635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:48.446635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:48.446635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:48.446635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:48.446635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:48.446635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:48.446635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:48.446635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:48.446635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:48.446635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:48.446635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:48.446635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:48.446635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:48.446635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:48.446635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:48.446635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:48.446635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:48.447551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:48.447551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:48.447551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:48.447551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:48.447551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 448/878] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:48.452567 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce551b-a1e4-4eae-a5da-7b2a6de72f17 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:48.573047 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce551b-a1e4-4eae-a5da-7b2a6de72f17 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:48.609660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c43e01-1e2e-485e-96c4-9e6bee301117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:48.610564 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c43e01-1e2e-485e-96c4-9e6bee301117 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:48.610997 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c43e01-1e2e-485e-96c4-9e6bee301117 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:48.611436 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c43e01-1e2e-485e-96c4-9e6bee301117 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:48.612317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c43e01-1e2e-485e-96c4-9e6bee301117 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:48.612793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c43e01-1e2e-485e-96c4-9e6bee301117 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:48.614326 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c43e01-1e2e-485e-96c4-9e6bee301117 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:48.627896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 431/879] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:48.633468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb209e7-2088-41ed-8ca4-30b4ba4cbd92 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:48.740913 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb209e7-2088-41ed-8ca4-30b4ba4cbd92 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:48.743341 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:48.777551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a2e87-9aad-4f0f-841b-4abf329489b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:48.778364 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a2e87-9aad-4f0f-841b-4abf329489b9 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:48.778643 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a2e87-9aad-4f0f-841b-4abf329489b9 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:48.778923 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a2e87-9aad-4f0f-841b-4abf329489b9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:48.786401 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a2e87-9aad-4f0f-841b-4abf329489b9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:48.787854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a2e87-9aad-4f0f-841b-4abf329489b9 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:48.803273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-5d6a2e87-9aad-4f0f-841b-4abf329489b9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:26:48.803273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:48.803273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:48.803273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:48.803273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:48.803273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:48.803273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:48.803273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:48.803273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:48.803273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:48.803273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:48.803273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:48.803273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:48.803273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:48.803273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:48.803273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:48.804279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:48.804279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:48.804279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:48.804279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:48.804279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:48.804279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:48.804279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:48.804279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:48.804279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:48.804279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:48.804279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:48.804279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:48.804279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:48.804279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:48.804279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:48.804279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:48.804279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:48.804279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:48.805288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:48.805288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:48.805288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:48.805288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:48.805288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:48.805288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:48.805288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:48.805288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:48.805288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:26:48.805288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:48.805288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 449/880] 10.222.0.71 () {56 vars in 1117 bytes} [Thu Jul 21 20:26:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:48.812111 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c43e01-1e2e-485e-96c4-9e6bee301117 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:48.864943 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:48.916740 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c43e01-1e2e-485e-96c4-9e6bee301117 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:48.952737 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f1056-dd4d-4aa9-892b-aae603c1d8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:48.953449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f1056-dd4d-4aa9-892b-aae603c1d8b4 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:48.953735 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f1056-dd4d-4aa9-892b-aae603c1d8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:48.954008 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f1056-dd4d-4aa9-892b-aae603c1d8b4 None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:48.954875 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f1056-dd4d-4aa9-892b-aae603c1d8b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:48.955187 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f1056-dd4d-4aa9-892b-aae603c1d8b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:48.956624 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f1056-dd4d-4aa9-892b-aae603c1d8b4 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:48.975524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 432/881] 10.222.0.71 () {56 vars in 1149 bytes} [Thu Jul 21 20:26:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:48.981696 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a2e87-9aad-4f0f-841b-4abf329489b9 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:49.088943 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a2e87-9aad-4f0f-841b-4abf329489b9 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:49.108749 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828deb9-c04b-450d-a1ca-a5199cd0e341 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:49.109112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828deb9-c04b-450d-a1ca-a5199cd0e341 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:49.109330 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828deb9-c04b-450d-a1ca-a5199cd0e341 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:49.110936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 433/882] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:49.116133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab768bff-37de-4094-a142-53347ef93517 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:49.116363 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab768bff-37de-4094-a142-53347ef93517 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:49.116615 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab768bff-37de-4094-a142-53347ef93517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:49.130215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd249cf-2f04-4d69-bcd7-a982910e18cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:49.131234 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd249cf-2f04-4d69-bcd7-a982910e18cf None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:49.131753 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd249cf-2f04-4d69-bcd7-a982910e18cf None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:49.132232 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd249cf-2f04-4d69-bcd7-a982910e18cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:49.139169 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-ab768bff-37de-4094-a142-53347ef93517 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:49.140626 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-ab768bff-37de-4094-a142-53347ef93517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:49.167049 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:49.209375 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd249cf-2f04-4d69-bcd7-a982910e18cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:49.211209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd249cf-2f04-4d69-bcd7-a982910e18cf None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:49.236379 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 434/883] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:49 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:26:49.258262 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab768bff-37de-4094-a142-53347ef93517 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:49.294516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:49.353112 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 450/884] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:26:48 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:26:49.354505 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd249cf-2f04-4d69-bcd7-a982910e18cf None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:49.380984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab768bff-37de-4094-a142-53347ef93517 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:49.423942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467959c1-473d-46ab-baea-74a9296565f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:49.424935 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467959c1-473d-46ab-baea-74a9296565f2 None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:49.425452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467959c1-473d-46ab-baea-74a9296565f2 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:49.425991 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467959c1-473d-46ab-baea-74a9296565f2 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:49.426873 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:49.428106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467959c1-473d-46ab-baea-74a9296565f2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:49.429733 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467959c1-473d-46ab-baea-74a9296565f2 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:49.450865 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 435/885] 10.222.0.71 () {60 vars in 1141 bytes} [Thu Jul 21 20:26:49 2022] POST /identity/v3/services => generated 269 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:26:49.496782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd249cf-2f04-4d69-bcd7-a982910e18cf None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:49.539318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fb62f-c8d3-4b58-968c-094ffe381c4d None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:49.539626 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fb62f-c8d3-4b58-968c-094ffe381c4d None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:49.539884 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fb62f-c8d3-4b58-968c-094ffe381c4d None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:49.545430 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ccb23a-2942-417a-8517-9e1eceb86a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:49.545635 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 436/886] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:26:49.546751 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccb23a-2942-417a-8517-9e1eceb86a53 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:49.547214 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccb23a-2942-417a-8517-9e1eceb86a53 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:49.547555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df1fa5-69b8-4b27-8db3-c6174b112c1a None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:49.547850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df1fa5-69b8-4b27-8db3-c6174b112c1a None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:49.547972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccb23a-2942-417a-8517-9e1eceb86a53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:49.548140 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df1fa5-69b8-4b27-8db3-c6174b112c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:49.560232 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ccb23a-2942-417a-8517-9e1eceb86a53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:49.561757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ccb23a-2942-417a-8517-9e1eceb86a53 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:49.568560 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-85df1fa5-69b8-4b27-8db3-c6174b112c1a None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:49.570147 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-85df1fa5-69b8-4b27-8db3-c6174b112c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:49.582453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-e0ccb23a-2942-417a-8517-9e1eceb86a53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:49.582453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:49.582453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:49.582453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:49.582453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:49.582453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:49.582453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:49.582453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:49.582453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:49.582453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:49.582453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:49.582453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:49.582453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:49.582453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:49.582453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:49.583424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:49.583424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:49.583424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:49.583424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:49.583424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:49.583424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:49.583424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:49.583424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:49.583424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:49.583424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:49.583424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:49.583424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:49.583424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:49.583424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:49.583424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:49.583424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:49.583424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:49.584320 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:49.584320 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:49.584320 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:49.584320 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:49.586853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 451/887] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:49.592824 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ccb23a-2942-417a-8517-9e1eceb86a53 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:49.656125 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 437/888] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:26:49.680850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df1fa5-69b8-4b27-8db3-c6174b112c1a None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:49.719158 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:49.720221 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ccb23a-2942-417a-8517-9e1eceb86a53 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:49.740067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:49.773456 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6ecd7-131f-4461-93c1-f0f11eb2b1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:49.774148 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6ecd7-131f-4461-93c1-f0f11eb2b1f3 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:49.774428 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6ecd7-131f-4461-93c1-f0f11eb2b1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:49.774701 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6ecd7-131f-4461-93c1-f0f11eb2b1f3 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:49.775442 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6ecd7-131f-4461-93c1-f0f11eb2b1f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:49.775727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6ecd7-131f-4461-93c1-f0f11eb2b1f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:49.777231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6ecd7-131f-4461-93c1-f0f11eb2b1f3 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:49.793441 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 452/889] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:49.802070 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6ecd7-131f-4461-93c1-f0f11eb2b1f3 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:49.811306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df1fa5-69b8-4b27-8db3-c6174b112c1a None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:49.863204 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d984255-4cc3-46ef-b33b-479f224f6962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:49.864799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d984255-4cc3-46ef-b33b-479f224f6962 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:49.865574 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d984255-4cc3-46ef-b33b-479f224f6962 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:49.866006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d984255-4cc3-46ef-b33b-479f224f6962 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:49.867101 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d984255-4cc3-46ef-b33b-479f224f6962 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:49.869417 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d984255-4cc3-46ef-b33b-479f224f6962 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:49.885116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-5d984255-4cc3-46ef-b33b-479f224f6962 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 20:26:49.885116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:49.885116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:26:49.885116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:26:49.885116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:26:49.885116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:26:49.885116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:26:49.885116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:49.885116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 20:26:49.885116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:49.885116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:49.885116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:49.885116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:49.885116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:49.885116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:49.885116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:49.885116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:49.886932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:49.886932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:49.886932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:49.886932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:49.886932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:49.886932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:49.886932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:26:49.886932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:26:49.886932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:26:49.886932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:26:49.886932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:49.886932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:49.886932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:49.886932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:49.886932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:49.886932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:49.887799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:49.887799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:49.887799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:49.887799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:49.887799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:49.887799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:49.887799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:49.887799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:49.887799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:49.887799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:49.887799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:26:49.887799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:49.887799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 20:26:49.887799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:49.888540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 438/890] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:26:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:26:49.893903 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d984255-4cc3-46ef-b33b-479f224f6962 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:49.930478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6ecd7-131f-4461-93c1-f0f11eb2b1f3 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:49.984980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859779d9-ed63-4309-8bde-640b97e420cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:49.986224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859779d9-ed63-4309-8bde-640b97e420cb None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:49.987659 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859779d9-ed63-4309-8bde-640b97e420cb None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:49.988078 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859779d9-ed63-4309-8bde-640b97e420cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:49.997844 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859779d9-ed63-4309-8bde-640b97e420cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:50.000171 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859779d9-ed63-4309-8bde-640b97e420cb None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:50.011491 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:50.022619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-859779d9-ed63-4309-8bde-640b97e420cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:50.022619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:50.022619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:50.022619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:50.022619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:50.022619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:50.022619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:50.022619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:50.022619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:50.022619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:50.022619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:50.022619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:50.022619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:50.022619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:50.022619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:50.023560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:50.023560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:50.023560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:50.023560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:50.023560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:50.023560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:50.023560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:50.023560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:50.023560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:50.023560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:50.023560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:50.023560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:50.023560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:50.023560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:50.023560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:50.023560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:50.023560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:50.024456 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:50.024456 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:50.024456 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:50.024456 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:50.024456 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:50.024456 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:50.024456 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:50.024456 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:50.024889 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d984255-4cc3-46ef-b33b-479f224f6962 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:50.028500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 453/891] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:50.034117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859779d9-ed63-4309-8bde-640b97e420cb None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:50.067174 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0260eb2-013f-41b2-93f7-935c686470be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:50.067878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0260eb2-013f-41b2-93f7-935c686470be None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:50.068165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0260eb2-013f-41b2-93f7-935c686470be None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:50.068442 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0260eb2-013f-41b2-93f7-935c686470be None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:50.069194 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0260eb2-013f-41b2-93f7-935c686470be None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:50.069513 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0260eb2-013f-41b2-93f7-935c686470be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:50.070912 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0260eb2-013f-41b2-93f7-935c686470be None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:50.076676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 439/892] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:26:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:50.078356 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85ace3-4b29-4d5d-8919-868252ab0ce4 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:50.078651 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85ace3-4b29-4d5d-8919-868252ab0ce4 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:50.078929 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85ace3-4b29-4d5d-8919-868252ab0ce4 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:50.080352 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 440/893] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:26:50.081772 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85ace3-4b29-4d5d-8919-868252ab0ce4 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:50.084741 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:50.160633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859779d9-ed63-4309-8bde-640b97e420cb None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:50.193721 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85ace3-4b29-4d5d-8919-868252ab0ce4 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:50.197569 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58ca4b-ea5a-47b9-be45-cf303a4629d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:50.198233 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58ca4b-ea5a-47b9-be45-cf303a4629d1 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:50.198518 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58ca4b-ea5a-47b9-be45-cf303a4629d1 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:50.198903 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58ca4b-ea5a-47b9-be45-cf303a4629d1 None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:50.199700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58ca4b-ea5a-47b9-be45-cf303a4629d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:50.200013 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58ca4b-ea5a-47b9-be45-cf303a4629d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:50.201657 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58ca4b-ea5a-47b9-be45-cf303a4629d1 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:50.215240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 454/894] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:50.216585 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440ab91-caf1-4592-a34d-c2a5a17d3529 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:50.216880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440ab91-caf1-4592-a34d-c2a5a17d3529 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:50.218287 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440ab91-caf1-4592-a34d-c2a5a17d3529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:50.230804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97d04e-632a-4217-ab0e-2efbf617c491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:50.231476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97d04e-632a-4217-ab0e-2efbf617c491 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:50.231766 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97d04e-632a-4217-ab0e-2efbf617c491 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:50.232041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97d04e-632a-4217-ab0e-2efbf617c491 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:50.232758 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97d04e-632a-4217-ab0e-2efbf617c491 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:50.233089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97d04e-632a-4217-ab0e-2efbf617c491 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:50.234460 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97d04e-632a-4217-ab0e-2efbf617c491 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:50.237268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-a440ab91-caf1-4592-a34d-c2a5a17d3529 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:50.238649 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a440ab91-caf1-4592-a34d-c2a5a17d3529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:50.239466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 441/895] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:26:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:50.240251 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97d04e-632a-4217-ab0e-2efbf617c491 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:50.267390 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:50.318944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 455/896] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:26:50.319726 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440ab91-caf1-4592-a34d-c2a5a17d3529 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:50.344547 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:50.378468 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97d04e-632a-4217-ab0e-2efbf617c491 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:50.415628 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba027ab7-1de2-4101-b4c2-bffe977c4315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:50.416282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba027ab7-1de2-4101-b4c2-bffe977c4315 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:50.416566 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba027ab7-1de2-4101-b4c2-bffe977c4315 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:50.416836 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba027ab7-1de2-4101-b4c2-bffe977c4315 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:50.417734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba027ab7-1de2-4101-b4c2-bffe977c4315 None admin] RBAC: Adding query filter params (role.id=4eefa64717a84653822b16532ebf6e75, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=b94e634e8855486a9ce14d3de9d986fb) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:50.418039 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba027ab7-1de2-4101-b4c2-bffe977c4315 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:50.419482 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba027ab7-1de2-4101-b4c2-bffe977c4315 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:50.433170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 442/897] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:26:50 2022] GET /identity/v3/role_assignments?user.id=b94e634e8855486a9ce14d3de9d986fb&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=4eefa64717a84653822b16532ebf6e75 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:50.433170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba027ab7-1de2-4101-b4c2-bffe977c4315 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:50.433575 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440ab91-caf1-4592-a34d-c2a5a17d3529 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:50.484456 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af8008-5d8b-4a63-9f3f-e9d7917f0efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:50.485193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af8008-5d8b-4a63-9f3f-e9d7917f0efb None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:50.485729 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af8008-5d8b-4a63-9f3f-e9d7917f0efb None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:50.486016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af8008-5d8b-4a63-9f3f-e9d7917f0efb None admin] PATH_INFO: `/v3/endpoints` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:50.486843 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af8008-5d8b-4a63-9f3f-e9d7917f0efb None admin] RBAC: Adding query filter params (interface=public, service_id=a17da32527844a78856ab8313b2ee291, region_id=RegionOne) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:50.487156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af8008-5d8b-4a63-9f3f-e9d7917f0efb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:50.488641 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af8008-5d8b-4a63-9f3f-e9d7917f0efb None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:50.500549 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 456/898] 10.222.0.71 () {56 vars in 1251 bytes} [Thu Jul 21 20:26:50 2022] GET /identity/v3/endpoints?service_id=a17da32527844a78856ab8313b2ee291&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:50.503308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af8008-5d8b-4a63-9f3f-e9d7917f0efb None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:50.554957 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:50.584150 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba027ab7-1de2-4101-b4c2-bffe977c4315 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:50.624630 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761e8e0-660c-435b-b539-44955dc1b37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:50.625850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761e8e0-660c-435b-b539-44955dc1b37e None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:50.625850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761e8e0-660c-435b-b539-44955dc1b37e None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:50.626041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761e8e0-660c-435b-b539-44955dc1b37e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:50.627977 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761e8e0-660c-435b-b539-44955dc1b37e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:50.628101 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761e8e0-660c-435b-b539-44955dc1b37e None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:50.632939 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:50.650817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af8008-5d8b-4a63-9f3f-e9d7917f0efb None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:50.652050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-f761e8e0-660c-435b-b539-44955dc1b37e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:26:50.652050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:50.652050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:26:50.652050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:26:50.652050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:26:50.652050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:26:50.652050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:26:50.652050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:50.652050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:26:50.652050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:50.652050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:50.652050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:50.652050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:50.652050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:50.652050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:50.652050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:50.652050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:50.653095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:50.653095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:50.653095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:50.653095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:50.653095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:50.653095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:50.653095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:26:50.653095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:26:50.653095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:26:50.653095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:26:50.653095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:50.653095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:50.653095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:50.653095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:50.653095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:50.653095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:50.653953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:50.653953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:50.653953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:50.653953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:50.653953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:50.653953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:50.653953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:50.653953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:50.653953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:50.653953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:50.653953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:26:50.653953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:50.653953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:26:50.653953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:50.654726 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 443/899] 10.222.0.71 () {56 vars in 1126 bytes} [Thu Jul 21 20:26:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:50.662054 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761e8e0-660c-435b-b539-44955dc1b37e None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:50.700472 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc08979-5caf-4a8b-a299-e42990353eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:50.702293 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc08979-5caf-4a8b-a299-e42990353eaf None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:50.702640 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc08979-5caf-4a8b-a299-e42990353eaf None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:50.702978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc08979-5caf-4a8b-a299-e42990353eaf None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:50.703777 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc08979-5caf-4a8b-a299-e42990353eaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:50.705419 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc08979-5caf-4a8b-a299-e42990353eaf None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:50.714610 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 457/900] 10.222.0.71 () {56 vars in 1087 bytes} [Thu Jul 21 20:26:50 2022] GET /identity/v3/services => generated 1268 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:26:50.789834 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761e8e0-660c-435b-b539-44955dc1b37e None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:50.830186 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222801d-f4b2-4664-86e1-0e79fcb6802d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:50.831246 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222801d-f4b2-4664-86e1-0e79fcb6802d None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:50.831737 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222801d-f4b2-4664-86e1-0e79fcb6802d None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:50.832166 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222801d-f4b2-4664-86e1-0e79fcb6802d None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:50.833112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222801d-f4b2-4664-86e1-0e79fcb6802d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:50.833618 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222801d-f4b2-4664-86e1-0e79fcb6802d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:50.835210 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222801d-f4b2-4664-86e1-0e79fcb6802d None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:50.842066 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 444/901] 10.222.0.71 () {56 vars in 1122 bytes} [Thu Jul 21 20:26:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:50.848259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc08979-5caf-4a8b-a299-e42990353eaf None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:50.954631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc08979-5caf-4a8b-a299-e42990353eaf None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:50.992516 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43834b9-4bd6-49bd-acc1-5d25c7b2230f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:50.993956 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43834b9-4bd6-49bd-acc1-5d25c7b2230f None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:50.994444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43834b9-4bd6-49bd-acc1-5d25c7b2230f None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:50.994886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43834b9-4bd6-49bd-acc1-5d25c7b2230f None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:50.995789 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43834b9-4bd6-49bd-acc1-5d25c7b2230f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:50.996263 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43834b9-4bd6-49bd-acc1-5d25c7b2230f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:50.997812 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43834b9-4bd6-49bd-acc1-5d25c7b2230f None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:51.003673 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 458/902] 10.222.0.71 () {56 vars in 1122 bytes} [Thu Jul 21 20:26:50 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:51.664905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad35d12-2106-40e3-b032-16be7c01ef7f None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:51.665484 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad35d12-2106-40e3-b032-16be7c01ef7f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:51.665484 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad35d12-2106-40e3-b032-16be7c01ef7f None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:51.667131 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 445/903] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:51.672518 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140de1d3-fc68-48c3-8e10-2522d39a60e6 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:51.672808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140de1d3-fc68-48c3-8e10-2522d39a60e6 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:51.673112 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140de1d3-fc68-48c3-8e10-2522d39a60e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:51.690520 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-140de1d3-fc68-48c3-8e10-2522d39a60e6 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:51.691718 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-140de1d3-fc68-48c3-8e10-2522d39a60e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:51.701736 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:51.769264 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 459/904] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:26:51.787167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad35d12-2106-40e3-b032-16be7c01ef7f None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:51.801180 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:51.901378 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad35d12-2106-40e3-b032-16be7c01ef7f None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:51.939904 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fbca1-8e80-462f-a451-f0e39d6ca10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:51.941100 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fbca1-8e80-462f-a451-f0e39d6ca10d None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:51.941634 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fbca1-8e80-462f-a451-f0e39d6ca10d None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:51.942131 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fbca1-8e80-462f-a451-f0e39d6ca10d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:51.943003 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832fbca1-8e80-462f-a451-f0e39d6ca10d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:51.944678 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832fbca1-8e80-462f-a451-f0e39d6ca10d None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:51.958444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-832fbca1-8e80-462f-a451-f0e39d6ca10d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 20:26:51.958444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:51.958444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:26:51.958444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:26:51.958444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:26:51.958444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:26:51.958444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:26:51.958444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:51.958444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 20:26:51.958444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:51.958444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:51.958444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:51.958444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:51.958444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:51.958444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:51.958444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:51.959453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:51.959453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:51.959453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:51.959453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:51.959453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:51.959453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:51.959453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:51.959453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:26:51.959453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:26:51.959453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:26:51.959453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:26:51.959453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:51.959453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:51.959453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:51.959453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:51.959453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:51.960312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:51.960312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:51.960312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:51.960312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:51.960312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:51.960312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:51.960312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:51.960312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:51.960312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:51.960312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:51.960312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:51.960312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:26:51.960312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:51.960312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 20:26:51.960312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:51.963243 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 446/905] 10.222.0.71 () {56 vars in 1132 bytes} [Thu Jul 21 20:26:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:51.969203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140de1d3-fc68-48c3-8e10-2522d39a60e6 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:52.077609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140de1d3-fc68-48c3-8e10-2522d39a60e6 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:52.113107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9747218-ce11-4c69-9be4-8a228c0135e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:52.113771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9747218-ce11-4c69-9be4-8a228c0135e3 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:52.114050 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9747218-ce11-4c69-9be4-8a228c0135e3 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:52.114321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9747218-ce11-4c69-9be4-8a228c0135e3 None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:52.115023 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9747218-ce11-4c69-9be4-8a228c0135e3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:52.115323 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9747218-ce11-4c69-9be4-8a228c0135e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:52.116686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9747218-ce11-4c69-9be4-8a228c0135e3 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:52.122057 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 460/906] 10.222.0.71 () {56 vars in 1126 bytes} [Thu Jul 21 20:26:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:52.127541 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fbca1-8e80-462f-a451-f0e39d6ca10d None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:52.209558 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:52.237003 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fbca1-8e80-462f-a451-f0e39d6ca10d None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:52.277420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3dfc3-045b-4fcf-8f8b-03b46e8c205d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:52.278498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3dfc3-045b-4fcf-8f8b-03b46e8c205d None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:52.279002 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3dfc3-045b-4fcf-8f8b-03b46e8c205d None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:52.279491 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3dfc3-045b-4fcf-8f8b-03b46e8c205d None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:52.280416 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3dfc3-045b-4fcf-8f8b-03b46e8c205d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:52.280948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3dfc3-045b-4fcf-8f8b-03b46e8c205d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:52.282537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3dfc3-045b-4fcf-8f8b-03b46e8c205d None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:52.288340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 447/907] 10.222.0.71 () {56 vars in 1126 bytes} [Thu Jul 21 20:26:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:52.294362 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9747218-ce11-4c69-9be4-8a228c0135e3 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:52.307397 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:52.400156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9747218-ce11-4c69-9be4-8a228c0135e3 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:52.434937 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872056a-e451-4da4-aa3c-2205436f2ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:52.435602 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872056a-e451-4da4-aa3c-2205436f2ad8 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:52.435928 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872056a-e451-4da4-aa3c-2205436f2ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:52.436206 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872056a-e451-4da4-aa3c-2205436f2ad8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:52.436995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872056a-e451-4da4-aa3c-2205436f2ad8 None admin] RBAC: Adding query filter params (interface=public, service_id=e5a77f2436314789a82e27ae95ff0901, region_id=RegionOne) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:52.437330 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872056a-e451-4da4-aa3c-2205436f2ad8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:52.438695 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872056a-e451-4da4-aa3c-2205436f2ad8 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:52.444301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 461/908] 10.222.0.71 () {56 vars in 1251 bytes} [Thu Jul 21 20:26:52 2022] GET /identity/v3/endpoints?service_id=e5a77f2436314789a82e27ae95ff0901&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:52.449804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3dfc3-045b-4fcf-8f8b-03b46e8c205d None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:52.563468 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3dfc3-045b-4fcf-8f8b-03b46e8c205d None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:52.599715 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefd1f4-584e-461a-a00c-616948bc5d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:52.600384 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefd1f4-584e-461a-a00c-616948bc5d26 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:52.600667 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefd1f4-584e-461a-a00c-616948bc5d26 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:52.600944 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefd1f4-584e-461a-a00c-616948bc5d26 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:52.601672 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adefd1f4-584e-461a-a00c-616948bc5d26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:52.603039 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adefd1f4-584e-461a-a00c-616948bc5d26 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:52.609901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 448/909] 10.222.0.71 () {56 vars in 1087 bytes} [Thu Jul 21 20:26:52 2022] GET /identity/v3/services => generated 1268 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:26:52.622069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb445de1-e884-4695-9c14-72491182aaeb None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:52.622387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb445de1-e884-4695-9c14-72491182aaeb None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:52.622620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb445de1-e884-4695-9c14-72491182aaeb None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:52.624321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 462/910] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:52.630283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca97d6a-850f-4233-a102-7bb1b030c6f2 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:52.630587 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca97d6a-850f-4233-a102-7bb1b030c6f2 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:52.630840 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca97d6a-850f-4233-a102-7bb1b030c6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:52.636662 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:52.651048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-8ca97d6a-850f-4233-a102-7bb1b030c6f2 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:52.652341 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-8ca97d6a-850f-4233-a102-7bb1b030c6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:52.731413 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 449/911] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:26:52.753713 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb445de1-e884-4695-9c14-72491182aaeb None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:52.820032 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba36814-acbf-4d54-902b-80b9395ea7e8 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:52.820374 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba36814-acbf-4d54-902b-80b9395ea7e8 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:52.820599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba36814-acbf-4d54-902b-80b9395ea7e8 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:52.822251 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 450/912] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:52.827902 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a19714-11fd-472e-ba8d-119beb124884 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:52.828187 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a19714-11fd-472e-ba8d-119beb124884 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:52.828467 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a19714-11fd-472e-ba8d-119beb124884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:52.835540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:52.852594 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-74a19714-11fd-472e-ba8d-119beb124884 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:52.853810 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-74a19714-11fd-472e-ba8d-119beb124884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:52.877574 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb445de1-e884-4695-9c14-72491182aaeb None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:52.924053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e985684-ad55-4bd0-97da-ff66ec803f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:52.925120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e985684-ad55-4bd0-97da-ff66ec803f56 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:52.925498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e985684-ad55-4bd0-97da-ff66ec803f56 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:52.925839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e985684-ad55-4bd0-97da-ff66ec803f56 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:52.931679 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 451/913] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:26:52.943527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e985684-ad55-4bd0-97da-ff66ec803f56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:52.953155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-0e985684-ad55-4bd0-97da-ff66ec803f56 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:26:52.953155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:52.953155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:52.953155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:52.953155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:52.953155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:52.953155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:52.953155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:52.953155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:52.953155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:52.953155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:52.953155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:52.953155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:52.953155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:52.953155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:52.953155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:52.954207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:52.954207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:52.954207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:52.954207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:52.954207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:52.954207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:52.954207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:52.954207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:52.954207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:52.954207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:52.954207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:52.954207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:52.954207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:52.954207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:52.954207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:52.954207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:52.955164 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:52.955164 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:52.955164 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:52.955164 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:52.955164 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:52.955164 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:52.955164 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:52.955164 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:52.955164 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:52.955164 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:52.955164 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:26:52.955164 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:52.957037 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a19714-11fd-472e-ba8d-119beb124884 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:52.958810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 463/914] 10.222.0.71 () {56 vars in 1120 bytes} [Thu Jul 21 20:26:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:52.966102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e985684-ad55-4bd0-97da-ff66ec803f56 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:53.042640 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:53.082632 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e985684-ad55-4bd0-97da-ff66ec803f56 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:53.095714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a19714-11fd-472e-ba8d-119beb124884 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:53.117243 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:53.123516 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1744a-1ac4-40a0-b755-acb168f69598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:53.124436 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1744a-1ac4-40a0-b755-acb168f69598 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:53.124861 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1744a-1ac4-40a0-b755-acb168f69598 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:53.125306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1744a-1ac4-40a0-b755-acb168f69598 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:53.126305 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1744a-1ac4-40a0-b755-acb168f69598 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:53.126882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1744a-1ac4-40a0-b755-acb168f69598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:53.128517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1744a-1ac4-40a0-b755-acb168f69598 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:53.143299 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b62ee-b704-4079-9daf-2629ff60a44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:53.143299 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b62ee-b704-4079-9daf-2629ff60a44b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:53.143589 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b62ee-b704-4079-9daf-2629ff60a44b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:53.143870 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b62ee-b704-4079-9daf-2629ff60a44b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:53.144093 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 464/915] 10.222.0.71 () {56 vars in 1115 bytes} [Thu Jul 21 20:26:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:53.144450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b62ee-b704-4079-9daf-2629ff60a44b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:53.145512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1744a-1ac4-40a0-b755-acb168f69598 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:53.145948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b62ee-b704-4079-9daf-2629ff60a44b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:53.166654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-8c2b62ee-b704-4079-9daf-2629ff60a44b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 20:26:53.166654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:53.166654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:26:53.166654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:26:53.166654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:26:53.166654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:26:53.166654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:26:53.166654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:53.166654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 20:26:53.166654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:53.166654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:53.166654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:53.166654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:53.166654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:53.166654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:53.166654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:53.166654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:53.168553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:53.168553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:53.168553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:53.168553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:53.168553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:53.168553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:53.168553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:26:53.168553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:26:53.168553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:26:53.168553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:26:53.168553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:53.168553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:53.168553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:53.168553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:53.168553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:53.168553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:53.170318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:53.170318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:53.170318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:53.170318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:53.170318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:53.170318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:53.170318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:53.170318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:53.170318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:53.170318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:53.170318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:26:53.170318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:53.170318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 20:26:53.170318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:53.170318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 452/916] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:26:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:53.172024 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ac2b3-7621-4baa-8193-3456a6f84fa4 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:53.172024 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ac2b3-7621-4baa-8193-3456a6f84fa4 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:53.172024 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ac2b3-7621-4baa-8193-3456a6f84fa4 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:53.173862 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 453/917] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:53.175437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ac2b3-7621-4baa-8193-3456a6f84fa4 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:53.258383 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1744a-1ac4-40a0-b755-acb168f69598 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:53.288027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ac2b3-7621-4baa-8193-3456a6f84fa4 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:53.291939 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5671d4-83c4-4559-8a53-9b958bd797a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:53.293306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5671d4-83c4-4559-8a53-9b958bd797a7 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:53.293588 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5671d4-83c4-4559-8a53-9b958bd797a7 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:53.293860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5671d4-83c4-4559-8a53-9b958bd797a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:53.304503 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5671d4-83c4-4559-8a53-9b958bd797a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:53.305987 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5671d4-83c4-4559-8a53-9b958bd797a7 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:53.317119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:53.322807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-7c5671d4-83c4-4559-8a53-9b958bd797a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:53.322807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:53.322807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:53.322807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:53.322807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:53.322807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:53.322807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:53.322807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:53.322807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:53.322807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:53.322807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:53.322807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:53.322807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:53.322807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:53.322807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:53.324813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:53.324813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:53.324813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:53.324813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:53.324813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:53.324813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:53.324813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:53.324813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:53.324813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:53.324813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:53.324813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:53.324813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:53.324813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:53.324813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:53.324813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:53.324813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:53.324813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:53.326748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:53.326748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:53.326748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:53.326748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:53.326748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 465/918] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:53.326748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc0b27-6355-44a3-8f20-478dd4f1ddb2 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:53.326748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc0b27-6355-44a3-8f20-478dd4f1ddb2 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:53.326748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc0b27-6355-44a3-8f20-478dd4f1ddb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:53.332405 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77642a1-898b-4dd0-a7ab-51083f2a3366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:53.333088 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77642a1-898b-4dd0-a7ab-51083f2a3366 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:53.333376 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77642a1-898b-4dd0-a7ab-51083f2a3366 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:53.333644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77642a1-898b-4dd0-a7ab-51083f2a3366 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:53.334367 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77642a1-898b-4dd0-a7ab-51083f2a3366 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:53.334721 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77642a1-898b-4dd0-a7ab-51083f2a3366 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:53.336035 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77642a1-898b-4dd0-a7ab-51083f2a3366 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:53.349032 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 454/919] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:26:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:53.349032 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77642a1-898b-4dd0-a7ab-51083f2a3366 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:53.357373 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-79bc0b27-6355-44a3-8f20-478dd4f1ddb2 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:53.359846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-79bc0b27-6355-44a3-8f20-478dd4f1ddb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:53.420163 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:53.447036 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 466/920] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:53 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:26:53.447783 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc0b27-6355-44a3-8f20-478dd4f1ddb2 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:53.490531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77642a1-898b-4dd0-a7ab-51083f2a3366 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:53.531570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621a243-5a2d-479d-bb56-a6606686f037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:53.532246 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621a243-5a2d-479d-bb56-a6606686f037 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:53.532529 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621a243-5a2d-479d-bb56-a6606686f037 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:53.532806 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621a243-5a2d-479d-bb56-a6606686f037 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:53.533581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621a243-5a2d-479d-bb56-a6606686f037 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:53.533891 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621a243-5a2d-479d-bb56-a6606686f037 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:53.535472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621a243-5a2d-479d-bb56-a6606686f037 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:53.553230 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 455/921] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:53.555297 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621a243-5a2d-479d-bb56-a6606686f037 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:53.584693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc0b27-6355-44a3-8f20-478dd4f1ddb2 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:53.618383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:53.638539 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913abb9-4927-4887-95ff-cec92ef2eba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:53.639219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913abb9-4927-4887-95ff-cec92ef2eba1 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:53.639497 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913abb9-4927-4887-95ff-cec92ef2eba1 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:53.639774 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913abb9-4927-4887-95ff-cec92ef2eba1 None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:53.640491 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3913abb9-4927-4887-95ff-cec92ef2eba1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:53.640798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3913abb9-4927-4887-95ff-cec92ef2eba1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:53.642219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3913abb9-4927-4887-95ff-cec92ef2eba1 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:53.659597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 467/922] 10.222.0.71 () {56 vars in 1114 bytes} [Thu Jul 21 20:26:53 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:53.660507 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913abb9-4927-4887-95ff-cec92ef2eba1 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:53.685930 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621a243-5a2d-479d-bb56-a6606686f037 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:53.723859 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:53.725754 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036326ce-f56f-4e48-9487-f3d0f5d7d99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:53.727416 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036326ce-f56f-4e48-9487-f3d0f5d7d99a None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:53.727774 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036326ce-f56f-4e48-9487-f3d0f5d7d99a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:53.728060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036326ce-f56f-4e48-9487-f3d0f5d7d99a None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:53.729178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036326ce-f56f-4e48-9487-f3d0f5d7d99a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:53.731082 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036326ce-f56f-4e48-9487-f3d0f5d7d99a None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:53.750145 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 456/923] 10.222.0.71 () {60 vars in 1141 bytes} [Thu Jul 21 20:26:53 2022] POST /identity/v3/services => generated 241 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:26:53.762663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036326ce-f56f-4e48-9487-f3d0f5d7d99a None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:53.803884 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913abb9-4927-4887-95ff-cec92ef2eba1 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:53.902236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78635053-b0c2-45c4-a92a-a40faf18001d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:53.903173 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78635053-b0c2-45c4-a92a-a40faf18001d None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:53.903461 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78635053-b0c2-45c4-a92a-a40faf18001d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:53.903733 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78635053-b0c2-45c4-a92a-a40faf18001d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:53.920767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78635053-b0c2-45c4-a92a-a40faf18001d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:53.922504 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78635053-b0c2-45c4-a92a-a40faf18001d None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:53.948217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-78635053-b0c2-45c4-a92a-a40faf18001d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:26:53.948217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:53.948217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:53.948217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:53.948217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:53.948217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:53.948217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:53.948217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:53.948217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:53.948217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:53.948217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:53.948217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:53.948217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:53.948217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:53.948217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:53.948217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:53.949838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:53.949838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:53.949838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:53.949838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:53.949838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:53.949838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:53.949838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:53.949838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:53.949838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:53.949838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:53.949838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:53.949838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:53.949838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:53.949838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:53.949838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:53.949838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:53.949838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:53.949838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:53.950926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:53.950926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:53.950926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:53.950926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:53.950926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:53.950926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:53.950926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:53.950926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:53.950926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:26:53.950926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:53.950926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 468/924] 10.222.0.71 () {56 vars in 1117 bytes} [Thu Jul 21 20:26:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:53.958735 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78635053-b0c2-45c4-a92a-a40faf18001d None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:53.966751 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036326ce-f56f-4e48-9487-f3d0f5d7d99a None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:53.985034 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:54.024451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e122d8-13ed-4ce2-bbc3-3aa480bb255b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:54.025803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e122d8-13ed-4ce2-bbc3-3aa480bb255b None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:54.026697 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e122d8-13ed-4ce2-bbc3-3aa480bb255b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:54.027228 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e122d8-13ed-4ce2-bbc3-3aa480bb255b None admin] PATH_INFO: `/v3/endpoints` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:54.028210 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e122d8-13ed-4ce2-bbc3-3aa480bb255b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:54.030347 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e122d8-13ed-4ce2-bbc3-3aa480bb255b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:54.063257 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 457/925] 10.222.0.71 () {60 vars in 1144 bytes} [Thu Jul 21 20:26:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:26:54.081654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:54.090276 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78635053-b0c2-45c4-a92a-a40faf18001d None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:54.139689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb4215-6269-4c6e-8597-3e9f0207bbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:54.140778 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb4215-6269-4c6e-8597-3e9f0207bbea None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:54.141316 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb4215-6269-4c6e-8597-3e9f0207bbea None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:54.141821 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb4215-6269-4c6e-8597-3e9f0207bbea None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:54.142908 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb4215-6269-4c6e-8597-3e9f0207bbea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:54.143448 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb4215-6269-4c6e-8597-3e9f0207bbea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:54.145196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb4215-6269-4c6e-8597-3e9f0207bbea None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:54.167059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 469/926] 10.222.0.71 () {56 vars in 1149 bytes} [Thu Jul 21 20:26:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:54.174239 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e122d8-13ed-4ce2-bbc3-3aa480bb255b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:54.290188 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e122d8-13ed-4ce2-bbc3-3aa480bb255b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:54.328864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b95aa-b497-4d6c-b46e-15b55fc8e6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:54.329791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b95aa-b497-4d6c-b46e-15b55fc8e6b4 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:54.330074 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b95aa-b497-4d6c-b46e-15b55fc8e6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:54.330352 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b95aa-b497-4d6c-b46e-15b55fc8e6b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:54.343452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b95aa-b497-4d6c-b46e-15b55fc8e6b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:54.344979 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b95aa-b497-4d6c-b46e-15b55fc8e6b4 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:54.363425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-987b95aa-b497-4d6c-b46e-15b55fc8e6b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:54.363425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:54.363425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:54.363425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:54.363425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:54.363425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:54.363425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:54.363425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:54.363425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:54.363425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:54.363425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:54.363425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:54.363425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:54.363425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:54.363425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:54.364394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:54.364394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:54.364394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:54.364394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:54.364394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:54.364394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:54.364394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:54.364394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:54.364394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:54.364394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:54.364394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:54.364394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:54.364394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:54.364394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:54.364394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:54.364394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:54.364394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:54.365352 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:54.365352 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:54.365352 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:54.365352 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:54.365352 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 458/927] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:54.371924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb4215-6269-4c6e-8597-3e9f0207bbea None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:54.480923 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb4215-6269-4c6e-8597-3e9f0207bbea None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:54.518056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db70063-8585-4d47-9680-c0eed6add474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:54.518723 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db70063-8585-4d47-9680-c0eed6add474 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:54.519005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db70063-8585-4d47-9680-c0eed6add474 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:54.519278 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db70063-8585-4d47-9680-c0eed6add474 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:54.520000 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db70063-8585-4d47-9680-c0eed6add474 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:54.520304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db70063-8585-4d47-9680-c0eed6add474 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:54.521771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db70063-8585-4d47-9680-c0eed6add474 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:54.534913 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 470/928] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:54.540704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b95aa-b497-4d6c-b46e-15b55fc8e6b4 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:54.566078 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:54.659115 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b95aa-b497-4d6c-b46e-15b55fc8e6b4 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:54.694917 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775650a5-f402-4659-bf94-aa44e0e2eddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:54.696079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775650a5-f402-4659-bf94-aa44e0e2eddf None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:54.696730 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775650a5-f402-4659-bf94-aa44e0e2eddf None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:54.697246 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775650a5-f402-4659-bf94-aa44e0e2eddf None admin] PATH_INFO: `/v3/projects/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:54.705315 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775650a5-f402-4659-bf94-aa44e0e2eddf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:54.707030 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775650a5-f402-4659-bf94-aa44e0e2eddf None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:54.722043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-775650a5-f402-4659-bf94-aa44e0e2eddf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:54.722043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:54.722043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:54.722043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:54.722043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:54.722043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:54.722043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:54.722043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:54.722043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:54.722043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:54.722043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:54.722043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:54.722043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:54.722043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:54.722043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:54.723050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:54.723050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:54.723050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:54.723050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:54.723050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:54.723050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:54.723050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:54.723050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:54.723050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:54.723050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:54.723050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:54.723050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:54.723050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:54.723050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:54.723050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:54.723050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:54.723050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:54.724293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:54.724293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:54.724293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:54.724293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:54.724293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:54.724293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:54.724293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:54.724293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:54.726783 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 459/929] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:54.732719 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db70063-8585-4d47-9680-c0eed6add474 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:54.746690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:54.764271 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de9fb8-6301-4ca2-a091-d43bd485e5d0 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:54.765230 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de9fb8-6301-4ca2-a091-d43bd485e5d0 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:54.765717 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de9fb8-6301-4ca2-a091-d43bd485e5d0 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:54.767608 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 460/930] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:54.773083 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f351b74-79c5-42fd-bde9-3a5586a797b6 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:54.773907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f351b74-79c5-42fd-bde9-3a5586a797b6 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:54.774394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f351b74-79c5-42fd-bde9-3a5586a797b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:54.794523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-2f351b74-79c5-42fd-bde9-3a5586a797b6 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:54.796062 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-2f351b74-79c5-42fd-bde9-3a5586a797b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:54.848994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db70063-8585-4d47-9680-c0eed6add474 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:54.869124 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 461/931] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:26:54.892484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161a58a-9fc2-4af0-a9a5-2664ce11d5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:54.893571 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161a58a-9fc2-4af0-a9a5-2664ce11d5cf None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:54.894068 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161a58a-9fc2-4af0-a9a5-2664ce11d5cf None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:54.894713 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161a58a-9fc2-4af0-a9a5-2664ce11d5cf None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:54.895951 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f351b74-79c5-42fd-bde9-3a5586a797b6 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:54.898078 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161a58a-9fc2-4af0-a9a5-2664ce11d5cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:54.898594 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161a58a-9fc2-4af0-a9a5-2664ce11d5cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:54.900498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161a58a-9fc2-4af0-a9a5-2664ce11d5cf None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:54.909026 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:54.915236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 471/932] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:54.921768 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161a58a-9fc2-4af0-a9a5-2664ce11d5cf None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:55.010173 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:55.025963 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f351b74-79c5-42fd-bde9-3a5586a797b6 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:55.043684 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161a58a-9fc2-4af0-a9a5-2664ce11d5cf None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:55.062829 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b64b3-7a34-4efe-99b4-df8e14aa7ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:55.063961 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b64b3-7a34-4efe-99b4-df8e14aa7ae7 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:55.064453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b64b3-7a34-4efe-99b4-df8e14aa7ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:55.064994 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b64b3-7a34-4efe-99b4-df8e14aa7ae7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:55.065950 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b64b3-7a34-4efe-99b4-df8e14aa7ae7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:55.067521 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b64b3-7a34-4efe-99b4-df8e14aa7ae7 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:55.079752 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24b9a2-ba8a-4b87-a331-1b010e091ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:55.080637 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24b9a2-ba8a-4b87-a331-1b010e091ba5 None admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:55.080917 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24b9a2-ba8a-4b87-a331-1b010e091ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:55.081232 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24b9a2-ba8a-4b87-a331-1b010e091ba5 None admin] PATH_INFO: `/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/b94e634e8855486a9ce14d3de9d986fb/roles/4eefa64717a84653822b16532ebf6e75` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:55.083004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-d41b64b3-7a34-4efe-99b4-df8e14aa7ae7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 20:26:55.083004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:55.083004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:26:55.083004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:26:55.083004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:26:55.083004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:26:55.083004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:26:55.083004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:55.083004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 20:26:55.083004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:55.083004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:55.083004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:55.083004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:55.083004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:55.083004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:55.083004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:55.084029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:55.084029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:55.084029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:55.084029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:55.084029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:55.084029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:55.084029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:55.084029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:26:55.084029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:26:55.084029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:26:55.084029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:26:55.084029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:55.084029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:55.084029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:55.084029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:55.084029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:55.084914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:55.084914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:55.084914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:55.084914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:55.084914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:55.084914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:55.084914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:55.084914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:55.084914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:55.084914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:55.084914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:55.084914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:26:55.084914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:55.084914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 20:26:55.084914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:55.085861 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a24b9a2-ba8a-4b87-a331-1b010e091ba5 None admin] RBAC: Authorizing `identity:create_grant(project_id=37e4afa595894a9994a8e6981ebb7c41, user_id=b94e634e8855486a9ce14d3de9d986fb, role_id=4eefa64717a84653822b16532ebf6e75)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:55.085861 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a24b9a2-ba8a-4b87-a331-1b010e091ba5 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:55.088091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 462/933] 10.222.0.71 () {56 vars in 1132 bytes} [Thu Jul 21 20:26:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:26:55.094131 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b64b3-7a34-4efe-99b4-df8e14aa7ae7 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:55.106770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 472/934] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:26:54 2022] PUT /identity/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/b94e634e8855486a9ce14d3de9d986fb/roles/4eefa64717a84653822b16532ebf6e75 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:26:55.182251 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:55.218238 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b64b3-7a34-4efe-99b4-df8e14aa7ae7 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:55.255258 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b8484-cc36-48ed-8795-279f1c4288ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:55.256266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b8484-cc36-48ed-8795-279f1c4288ce None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:55.256767 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b8484-cc36-48ed-8795-279f1c4288ce None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:55.257245 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b8484-cc36-48ed-8795-279f1c4288ce None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:55.258183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b8484-cc36-48ed-8795-279f1c4288ce None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:55.258677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b8484-cc36-48ed-8795-279f1c4288ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:55.260425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b8484-cc36-48ed-8795-279f1c4288ce None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:55.267062 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 463/935] 10.222.0.71 () {56 vars in 1126 bytes} [Thu Jul 21 20:26:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:55.273478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24b9a2-ba8a-4b87-a331-1b010e091ba5 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:55.391451 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24b9a2-ba8a-4b87-a331-1b010e091ba5 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:55.420770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:55.428690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868167d6-fe66-44d2-b14a-53808ff309fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:55.429473 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868167d6-fe66-44d2-b14a-53808ff309fd None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:55.429756 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868167d6-fe66-44d2-b14a-53808ff309fd None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:55.430036 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868167d6-fe66-44d2-b14a-53808ff309fd None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:55.430907 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868167d6-fe66-44d2-b14a-53808ff309fd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:55.431548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868167d6-fe66-44d2-b14a-53808ff309fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:55.433647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868167d6-fe66-44d2-b14a-53808ff309fd None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:55.439883 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 473/936] 10.222.0.71 () {56 vars in 1126 bytes} [Thu Jul 21 20:26:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:55.445473 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b8484-cc36-48ed-8795-279f1c4288ce None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:55.553249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b8484-cc36-48ed-8795-279f1c4288ce None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:55.588718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581ad74-2550-477c-b3b6-4b4539c780f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:55.589845 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581ad74-2550-477c-b3b6-4b4539c780f4 None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:55.590374 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581ad74-2550-477c-b3b6-4b4539c780f4 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:55.590856 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581ad74-2550-477c-b3b6-4b4539c780f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:55.591821 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581ad74-2550-477c-b3b6-4b4539c780f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:55.593421 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581ad74-2550-477c-b3b6-4b4539c780f4 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:55.601547 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:55.617119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 464/937] 10.222.0.71 () {60 vars in 1144 bytes} [Thu Jul 21 20:26:55 2022] POST /identity/v3/endpoints => generated 347 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:26:55.957860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c00f2-2df1-47d3-a582-a80d68b412b6 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:55.958378 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c00f2-2df1-47d3-a582-a80d68b412b6 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:55.958765 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c00f2-2df1-47d3-a582-a80d68b412b6 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:55.960524 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 474/938] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:55.966602 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c9979-f090-4875-838c-ec4a702f987e None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:55.967095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c9979-f090-4875-838c-ec4a702f987e None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:55.967355 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c9979-f090-4875-838c-ec4a702f987e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:55.985205 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-e03c9979-f090-4875-838c-ec4a702f987e None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:55.986409 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-e03c9979-f090-4875-838c-ec4a702f987e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:56.069193 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 465/939] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:26:56.088898 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c00f2-2df1-47d3-a582-a80d68b412b6 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:56.200762 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c00f2-2df1-47d3-a582-a80d68b412b6 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:56.234196 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403c4c6-4f1d-4913-b84c-c58b657816d0 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:56.234372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403c4c6-4f1d-4913-b84c-c58b657816d0 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:56.234656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403c4c6-4f1d-4913-b84c-c58b657816d0 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:56.236305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 466/940] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:26:56.238317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460e587-2539-447d-95a1-3ec19ff9bb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:56.239358 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460e587-2539-447d-95a1-3ec19ff9bb0a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:56.239840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460e587-2539-447d-95a1-3ec19ff9bb0a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:56.240340 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460e587-2539-447d-95a1-3ec19ff9bb0a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:56.241234 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460e587-2539-447d-95a1-3ec19ff9bb0a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:56.241357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdedf4-7fff-4b80-8277-4623dd41ea82 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:56.241634 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdedf4-7fff-4b80-8277-4623dd41ea82 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:56.241910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdedf4-7fff-4b80-8277-4623dd41ea82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:56.243461 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460e587-2539-447d-95a1-3ec19ff9bb0a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:56.259012 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-8ebdedf4-7fff-4b80-8277-4623dd41ea82 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:56.259304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-9460e587-2539-447d-95a1-3ec19ff9bb0a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:26:56.259304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:56.259304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:26:56.259304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:26:56.259304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:26:56.259304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:26:56.259304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:26:56.259304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:56.259304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:26:56.259304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:56.259304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:56.259304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:56.259304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:56.259304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:56.259304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:56.259304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:56.259304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:56.260319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:56.260319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:56.260319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:56.260319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:56.260319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:56.260319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:56.260319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:26:56.260319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:26:56.260319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:26:56.260319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:26:56.260319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:56.260319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:56.260319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:56.260319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:56.260319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:56.260319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:56.261321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:56.261321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:56.261321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:56.261321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:56.261321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:56.261321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:56.261321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:56.261321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:56.261321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:56.261321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:56.261321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:26:56.261321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:56.261321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:26:56.261321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:56.262077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-8ebdedf4-7fff-4b80-8277-4623dd41ea82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:56.264927 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 475/941] 10.222.0.71 () {56 vars in 1126 bytes} [Thu Jul 21 20:26:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:56.274216 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460e587-2539-447d-95a1-3ec19ff9bb0a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:56.337400 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 467/942] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:26:56.341576 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:56.357327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdedf4-7fff-4b80-8277-4623dd41ea82 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:56.382998 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460e587-2539-447d-95a1-3ec19ff9bb0a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:56.409955 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:56.421828 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485932e8-0a09-409c-a2c7-5a2196b7bc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:56.422784 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485932e8-0a09-409c-a2c7-5a2196b7bc21 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:56.423219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485932e8-0a09-409c-a2c7-5a2196b7bc21 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:56.423645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485932e8-0a09-409c-a2c7-5a2196b7bc21 None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:56.424523 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485932e8-0a09-409c-a2c7-5a2196b7bc21 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:56.424981 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485932e8-0a09-409c-a2c7-5a2196b7bc21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:56.426552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485932e8-0a09-409c-a2c7-5a2196b7bc21 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:56.432113 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 476/943] 10.222.0.71 () {56 vars in 1122 bytes} [Thu Jul 21 20:26:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:56.437693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485932e8-0a09-409c-a2c7-5a2196b7bc21 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:56.480221 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdedf4-7fff-4b80-8277-4623dd41ea82 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:56.516616 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e7de3-3f34-4208-9e23-88af9c3ebf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:56.517639 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e7de3-3f34-4208-9e23-88af9c3ebf16 None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:56.518122 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e7de3-3f34-4208-9e23-88af9c3ebf16 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:56.518593 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e7de3-3f34-4208-9e23-88af9c3ebf16 None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:56.519573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e7de3-3f34-4208-9e23-88af9c3ebf16 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:56.521310 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e7de3-3f34-4208-9e23-88af9c3ebf16 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:56.522922 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: WARNING keystone.server.flask.common [None req-660e7de3-3f34-4208-9e23-88af9c3ebf16 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 21 20:26:56.543033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.sql.core [None req-660e7de3-3f34-4208-9e23-88af9c3ebf16 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 20:26:56.543033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 20:26:56.543033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [parameters: {'id': 'ce68401527f4479895af304b998f4009', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 20:26:56.543033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=103664) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 20:26:56.548709 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485932e8-0a09-409c-a2c7-5a2196b7bc21 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:56.552106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-660e7de3-3f34-4208-9e23-88af9c3ebf16 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ce68401527f4479895af304b998f4009.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ce68401527f4479895af304b998f4009. Jul 21 20:26:56.552106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:56.552106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 20:26:56.552106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 20:26:56.552106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 20:26:56.552106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 20:26:56.552106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 20:26:56.552106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 20:26:56.552106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 20:26:56.552106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application conn.query(q) Jul 21 20:26:56.552106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 20:26:56.552106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 20:26:56.552106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 20:26:56.552924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application result.read() Jul 21 20:26:56.552924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 20:26:56.552924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 20:26:56.552924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 20:26:56.552924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 20:26:56.552924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 20:26:56.552924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 20:26:56.552924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 20:26:56.552924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 20:26:56.552924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 20:26:56.552924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:56.552924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 20:26:56.552924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:56.552924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:56.552924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 20:26:56.552924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 20:26:56.552924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 21 20:26:56.552924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 21 20:26:56.553916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 20:26:56.553916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application next(self.gen) Jul 21 20:26:56.553916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 20:26:56.553916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application yield resource Jul 21 20:26:56.553916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 20:26:56.553916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application next(self.gen) Jul 21 20:26:56.553916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 20:26:56.553916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 20:26:56.553916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 20:26:56.553916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 20:26:56.553916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 20:26:56.553916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise self.value Jul 21 20:26:56.553916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 20:26:56.553916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 20:26:56.553916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 20:26:56.553916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application session.commit() Jul 21 20:26:56.553916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 20:26:56.554812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 20:26:56.554812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 20:26:56.554812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 20:26:56.554812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 20:26:56.554812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self.session.flush() Jul 21 20:26:56.554812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 20:26:56.554812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 20:26:56.554812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 20:26:56.554812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 20:26:56.554812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 20:26:56.554812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application compat.raise_( Jul 21 20:26:56.554812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 20:26:56.554812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception Jul 21 20:26:56.554812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 20:26:56.554812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 20:26:56.554812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 20:26:56.554812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 20:26:56.555744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 20:26:56.555744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 20:26:56.555744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 20:26:56.555744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 20:26:56.555744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 20:26:56.555744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 20:26:56.555744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 20:26:56.555744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 20:26:56.555744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 20:26:56.555744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 20:26:56.555744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 20:26:56.555744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 20:26:56.555744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 20:26:56.555744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 20:26:56.555744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 20:26:56.556613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 20:26:56.556613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 20:26:56.556613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception Jul 21 20:26:56.556613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 20:26:56.556613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 20:26:56.556613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 20:26:56.556613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 20:26:56.556613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 20:26:56.556613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 20:26:56.556613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 20:26:56.556613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application conn.query(q) Jul 21 20:26:56.556613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 20:26:56.556613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 20:26:56.556613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 20:26:56.556613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application result.read() Jul 21 20:26:56.556613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 20:26:56.556613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 20:26:56.564646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 20:26:56.564646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 20:26:56.564646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 20:26:56.564646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 20:26:56.564646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 20:26:56.564646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 20:26:56.564646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 20:26:56.564646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 20:26:56.564646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [parameters: {'id': 'ce68401527f4479895af304b998f4009', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 20:26:56.564646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 20:26:56.564646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:56.564646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:56.564646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:56.564646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:56.564646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 21 20:26:56.567041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 21 20:26:56.567041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 20:26:56.567041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 20:26:56.567041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 21 20:26:56.567041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:56.567041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:56.567041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:56.567041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:56.567041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:56.567041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:56.567041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:56.567041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:56.567041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:56.567041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:56.567041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:56.567041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:56.567041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:56.569331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:56.569331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 21 20:26:56.569331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 21 20:26:56.569331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:56.569331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:56.569331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 21 20:26:56.569331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 21 20:26:56.569331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ce68401527f4479895af304b998f4009. Jul 21 20:26:56.569331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:56.569331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 468/944] 10.222.0.71 () {60 vars in 1140 bytes} [Thu Jul 21 20:26:56 2022] POST /identity/v3/projects => generated 262 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:26:56.570664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e7de3-3f34-4208-9e23-88af9c3ebf16 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:56.572933 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:56.584909 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21abe94-c2aa-4b77-b6e2-732ffe58ac0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:56.585679 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21abe94-c2aa-4b77-b6e2-732ffe58ac0b None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:56.586887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21abe94-c2aa-4b77-b6e2-732ffe58ac0b None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:56.586887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21abe94-c2aa-4b77-b6e2-732ffe58ac0b None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:56.587202 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21abe94-c2aa-4b77-b6e2-732ffe58ac0b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:56.587364 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21abe94-c2aa-4b77-b6e2-732ffe58ac0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:56.588621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21abe94-c2aa-4b77-b6e2-732ffe58ac0b None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:56.598757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 477/945] 10.222.0.71 () {56 vars in 1122 bytes} [Thu Jul 21 20:26:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:56.602194 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21abe94-c2aa-4b77-b6e2-732ffe58ac0b None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:56.705815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e7de3-3f34-4208-9e23-88af9c3ebf16 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:56.706297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21abe94-c2aa-4b77-b6e2-732ffe58ac0b None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:56.709201 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:56.744771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4901c-2025-4fa8-8384-0926705040f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:56.745590 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4901c-2025-4fa8-8384-0926705040f9 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:56.745875 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4901c-2025-4fa8-8384-0926705040f9 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:56.746194 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4901c-2025-4fa8-8384-0926705040f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:56.747003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4901c-2025-4fa8-8384-0926705040f9 None admin] RBAC: Adding query filter params (interface=public, service_id=92e0d2d675ed48d1811a544e7d184728, region_id=RegionOne) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:56.747316 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4901c-2025-4fa8-8384-0926705040f9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:56.748494 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ea6c3-26b3-4b06-928a-4922b8c9310c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:56.748799 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4901c-2025-4fa8-8384-0926705040f9 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:56.749737 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ea6c3-26b3-4b06-928a-4922b8c9310c None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:56.750351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ea6c3-26b3-4b06-928a-4922b8c9310c None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:56.750847 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ea6c3-26b3-4b06-928a-4922b8c9310c None admin] PATH_INFO: `/v3/projects/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:56.759183 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 478/946] 10.222.0.71 () {56 vars in 1251 bytes} [Thu Jul 21 20:26:56 2022] GET /identity/v3/endpoints?service_id=92e0d2d675ed48d1811a544e7d184728&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:56.760820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4901c-2025-4fa8-8384-0926705040f9 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:56.762224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ea6c3-26b3-4b06-928a-4922b8c9310c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:56.764450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ea6c3-26b3-4b06-928a-4922b8c9310c None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:56.782340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-f21ea6c3-26b3-4b06-928a-4922b8c9310c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:56.782340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:56.782340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:56.782340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:56.782340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:56.782340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:56.782340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:56.782340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:56.782340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:56.782340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:56.782340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:56.782340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:56.782340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:56.782340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:56.782340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:56.783264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:56.783264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:56.783264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:56.783264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:56.783264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:56.783264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:56.783264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:56.783264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:56.783264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:56.783264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:56.783264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:56.783264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:56.783264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:56.783264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:56.783264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:56.783264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:56.783264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:56.784183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:56.784183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:56.784183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:56.784183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:56.784183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:56.784183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:56.784183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:56.784183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:56.784183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 469/947] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:56.793432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ea6c3-26b3-4b06-928a-4922b8c9310c None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:56.823659 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:56.882021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4901c-2025-4fa8-8384-0926705040f9 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:56.900991 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ea6c3-26b3-4b06-928a-4922b8c9310c None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:56.920624 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd848b-5cfa-4f3e-a61d-ea7d443ddcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:56.921690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd848b-5cfa-4f3e-a61d-ea7d443ddcae None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:56.922175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd848b-5cfa-4f3e-a61d-ea7d443ddcae None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:56.922709 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd848b-5cfa-4f3e-a61d-ea7d443ddcae None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:56.923611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd848b-5cfa-4f3e-a61d-ea7d443ddcae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:56.925293 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd848b-5cfa-4f3e-a61d-ea7d443ddcae None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:56.935460 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 479/948] 10.222.0.71 () {56 vars in 1087 bytes} [Thu Jul 21 20:26:56 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:26:56.939817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bdfc7-954d-4c16-9de8-e9ce8e839d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:56.940844 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bdfc7-954d-4c16-9de8-e9ce8e839d51 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:56.941403 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bdfc7-954d-4c16-9de8-e9ce8e839d51 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:56.941886 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bdfc7-954d-4c16-9de8-e9ce8e839d51 None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:56.942914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541bdfc7-954d-4c16-9de8-e9ce8e839d51 None admin] RBAC: Adding query filter params (name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:56.943449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541bdfc7-954d-4c16-9de8-e9ce8e839d51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:56.945216 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541bdfc7-954d-4c16-9de8-e9ce8e839d51 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:56.969719 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 470/949] 10.222.0.71 () {56 vars in 1112 bytes} [Thu Jul 21 20:26:56 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:57.316184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ce17d-ba1d-498e-bdfd-9d3a00564d81 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:57.316704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ce17d-ba1d-498e-bdfd-9d3a00564d81 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:57.316704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ce17d-ba1d-498e-bdfd-9d3a00564d81 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:57.318394 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 480/950] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:57.324324 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48fffd-18a8-49df-8c2a-2f557bfc2c86 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:57.324727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48fffd-18a8-49df-8c2a-2f557bfc2c86 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:57.324952 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48fffd-18a8-49df-8c2a-2f557bfc2c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:57.342818 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-5a48fffd-18a8-49df-8c2a-2f557bfc2c86 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:57.344320 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-5a48fffd-18a8-49df-8c2a-2f557bfc2c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:57.367053 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:57.425439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 471/951] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:26:57.445156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ce17d-ba1d-498e-bdfd-9d3a00564d81 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:57.567476 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ce17d-ba1d-498e-bdfd-9d3a00564d81 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:57.581690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:57.603544 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcc212-fcba-48df-b05e-c9b54a34f7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:57.604360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcc212-fcba-48df-b05e-c9b54a34f7f0 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:57.604703 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcc212-fcba-48df-b05e-c9b54a34f7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:57.605481 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcc212-fcba-48df-b05e-c9b54a34f7f0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:57.616331 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcc212-fcba-48df-b05e-c9b54a34f7f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:26:57.625098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-1adcc212-fcba-48df-b05e-c9b54a34f7f0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:26:57.625098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:57.625098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:57.625098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:57.625098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:57.625098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:57.625098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:57.625098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:57.625098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:57.625098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:57.625098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:57.625098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:57.625098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:26:57.625098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:26:57.625098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:26:57.625098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:26:57.627259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:26:57.627259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:26:57.627259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:26:57.627259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:26:57.627259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:57.627259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:57.627259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:57.627259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:57.627259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:57.627259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:57.627259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:57.627259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:57.627259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:57.627259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:57.627259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:57.627259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:57.629372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:57.629372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:57.629372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:57.629372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:57.629372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:26:57.629372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:26:57.629372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:26:57.629372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:26:57.629372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:26:57.629372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:26:57.629372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:26:57.629372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:57.629372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 481/952] 10.222.0.71 () {56 vars in 1120 bytes} [Thu Jul 21 20:26:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:57.634438 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48fffd-18a8-49df-8c2a-2f557bfc2c86 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:57.736952 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48fffd-18a8-49df-8c2a-2f557bfc2c86 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:57.759388 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019dbdc4-115c-435c-b01f-fcb8f17ed51d None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:57.759746 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019dbdc4-115c-435c-b01f-fcb8f17ed51d None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:57.759922 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019dbdc4-115c-435c-b01f-fcb8f17ed51d None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:57.761720 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 482/953] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:57.766857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b25e44-6466-41fa-aab8-06eae9b6f20a None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:57.767140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b25e44-6466-41fa-aab8-06eae9b6f20a None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:57.767414 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b25e44-6466-41fa-aab8-06eae9b6f20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:57.773465 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f25b57-7110-4fd7-8b23-06e5d1df54ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:57.774952 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f25b57-7110-4fd7-8b23-06e5d1df54ec None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:57.775234 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f25b57-7110-4fd7-8b23-06e5d1df54ec None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:57.775509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f25b57-7110-4fd7-8b23-06e5d1df54ec None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:57.776325 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f25b57-7110-4fd7-8b23-06e5d1df54ec None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:57.777288 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f25b57-7110-4fd7-8b23-06e5d1df54ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:57.778653 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f25b57-7110-4fd7-8b23-06e5d1df54ec None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:57.784706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-f5b25e44-6466-41fa-aab8-06eae9b6f20a None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:57.785941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f5b25e44-6466-41fa-aab8-06eae9b6f20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:57.789478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 472/954] 10.222.0.71 () {56 vars in 1115 bytes} [Thu Jul 21 20:26:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:57.797426 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f25b57-7110-4fd7-8b23-06e5d1df54ec None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:57.821984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:57.869415 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 483/955] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:26:57.885622 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b25e44-6466-41fa-aab8-06eae9b6f20a None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:57.928090 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f25b57-7110-4fd7-8b23-06e5d1df54ec None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:57.969309 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a0ffe-f75b-42b2-a5bb-67ad8d784c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:57.970108 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a0ffe-f75b-42b2-a5bb-67ad8d784c71 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:57.971272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a0ffe-f75b-42b2-a5bb-67ad8d784c71 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:57.971272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a0ffe-f75b-42b2-a5bb-67ad8d784c71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:57.983718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a0ffe-f75b-42b2-a5bb-67ad8d784c71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:57.985260 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a0ffe-f75b-42b2-a5bb-67ad8d784c71 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:58.003973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-c69a0ffe-f75b-42b2-a5bb-67ad8d784c71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:58.003973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:58.003973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:58.003973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:58.003973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:58.003973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:58.003973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:58.003973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:58.003973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:58.003973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:58.003973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:58.003973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:58.003973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:58.003973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:58.003973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:58.006089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:58.006089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:58.006089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:58.006089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:58.006089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:58.006089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:58.006089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:58.006089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:58.006089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:58.006089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:58.006089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:58.006089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:58.006089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:58.006089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:58.006089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:58.006089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:58.006089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:58.008265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:58.008265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:58.008265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:58.008265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:58.008780 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b25e44-6466-41fa-aab8-06eae9b6f20a None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:58.017105 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 473/956] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:58.017105 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a0ffe-f75b-42b2-a5bb-67ad8d784c71 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:58.017527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:58.056251 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f0325-6a6c-4150-900c-2da0558f79b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:58.057052 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f0325-6a6c-4150-900c-2da0558f79b3 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:58.057377 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f0325-6a6c-4150-900c-2da0558f79b3 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:58.058141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f0325-6a6c-4150-900c-2da0558f79b3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:58.058803 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f0325-6a6c-4150-900c-2da0558f79b3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:58.060939 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f0325-6a6c-4150-900c-2da0558f79b3 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:58.077677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-190f0325-6a6c-4150-900c-2da0558f79b3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 20:26:58.077677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:58.077677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:26:58.077677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:26:58.077677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:26:58.077677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:26:58.077677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:26:58.077677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:58.077677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 20:26:58.077677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:58.077677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:58.077677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:58.077677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:58.077677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:58.077677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:58.077677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:58.077677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:58.078734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:58.078734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:58.078734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:58.078734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:58.078734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:58.078734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:58.078734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:26:58.078734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:26:58.078734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:26:58.078734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:26:58.078734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:58.078734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:58.078734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:58.078734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:58.078734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:58.078734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:58.079609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:58.079609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:58.079609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:58.079609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:58.079609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:58.079609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:58.079609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:58.079609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:58.079609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:58.079609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:58.079609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:26:58.079609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:58.079609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 20:26:58.079609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:58.081355 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 484/957] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:58.084721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f0325-6a6c-4150-900c-2da0558f79b3 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:58.092617 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:58.146328 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a0ffe-f75b-42b2-a5bb-67ad8d784c71 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:58.193645 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba6a93-f41c-456e-bb21-9a3786066546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:58.195468 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba6a93-f41c-456e-bb21-9a3786066546 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:58.196138 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba6a93-f41c-456e-bb21-9a3786066546 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:58.196724 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba6a93-f41c-456e-bb21-9a3786066546 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:58.197712 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba6a93-f41c-456e-bb21-9a3786066546 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:58.198210 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba6a93-f41c-456e-bb21-9a3786066546 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:58.199170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f0325-6a6c-4150-900c-2da0558f79b3 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:58.200605 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba6a93-f41c-456e-bb21-9a3786066546 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:58.219207 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 474/958] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:58.224430 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba6a93-f41c-456e-bb21-9a3786066546 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:58.246198 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf22f3c-751e-459c-a921-9174f9a97324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:58.247715 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf22f3c-751e-459c-a921-9174f9a97324 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:58.248198 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf22f3c-751e-459c-a921-9174f9a97324 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:58.248935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf22f3c-751e-459c-a921-9174f9a97324 None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:58.250137 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf22f3c-751e-459c-a921-9174f9a97324 None admin] RBAC: Adding query filter params (name=compute) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:58.251052 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf22f3c-751e-459c-a921-9174f9a97324 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:58.253175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf22f3c-751e-459c-a921-9174f9a97324 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:58.260383 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 485/959] 10.222.0.71 () {56 vars in 1112 bytes} [Thu Jul 21 20:26:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:58.265670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf22f3c-751e-459c-a921-9174f9a97324 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:58.308636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:58.333326 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba6a93-f41c-456e-bb21-9a3786066546 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:58.368936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:58.377647 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeea4b8-e402-4b17-af13-c471de4747bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:58.378542 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeea4b8-e402-4b17-af13-c471de4747bf None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:58.378918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeea4b8-e402-4b17-af13-c471de4747bf None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:58.379268 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeea4b8-e402-4b17-af13-c471de4747bf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:58.387010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeea4b8-e402-4b17-af13-c471de4747bf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:58.388550 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeea4b8-e402-4b17-af13-c471de4747bf None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:58.396211 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf22f3c-751e-459c-a921-9174f9a97324 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:58.404591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-1aeea4b8-e402-4b17-af13-c471de4747bf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:26:58.404591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:58.404591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:58.404591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:58.404591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:58.404591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:58.404591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:58.404591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:58.404591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:58.404591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:58.404591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:58.404591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:58.404591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:58.404591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:58.404591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:58.404591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:58.405546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:58.405546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:58.405546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:58.405546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:58.405546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:58.405546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:58.405546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:58.405546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:58.405546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:58.405546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:58.405546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:58.405546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:58.405546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:58.405546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:58.405546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:58.405546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:58.405546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:58.405546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:58.406498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:58.406498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:58.406498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:58.406498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:58.406498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:58.406498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:58.406498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:58.406498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:58.406498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:26:58.406498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:58.410560 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 475/960] 10.222.0.71 () {56 vars in 1117 bytes} [Thu Jul 21 20:26:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:58.411468 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeea4b8-e402-4b17-af13-c471de4747bf None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:58.440487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de1cf6-dc5b-44ec-a155-b0a04bc4d902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:58.442006 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de1cf6-dc5b-44ec-a155-b0a04bc4d902 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:58.442495 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de1cf6-dc5b-44ec-a155-b0a04bc4d902 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:58.442946 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de1cf6-dc5b-44ec-a155-b0a04bc4d902 None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:58.443987 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de1cf6-dc5b-44ec-a155-b0a04bc4d902 None admin] RBAC: Adding query filter params (type=compute) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:58.444738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de1cf6-dc5b-44ec-a155-b0a04bc4d902 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:58.446539 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de1cf6-dc5b-44ec-a155-b0a04bc4d902 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:58.459656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 486/961] 10.222.0.71 () {56 vars in 1112 bytes} [Thu Jul 21 20:26:58 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:58.531999 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeea4b8-e402-4b17-af13-c471de4747bf None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:58.571994 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f54cf-e6c4-4c48-8490-8257512eaacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:58.573134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f54cf-e6c4-4c48-8490-8257512eaacf None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:58.574156 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f54cf-e6c4-4c48-8490-8257512eaacf None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:58.574658 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f54cf-e6c4-4c48-8490-8257512eaacf None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:58.575728 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f54cf-e6c4-4c48-8490-8257512eaacf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:58.576247 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f54cf-e6c4-4c48-8490-8257512eaacf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:58.577965 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f54cf-e6c4-4c48-8490-8257512eaacf None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:58.599432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 476/962] 10.222.0.71 () {56 vars in 1149 bytes} [Thu Jul 21 20:26:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:58.605548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de1cf6-dc5b-44ec-a155-b0a04bc4d902 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:58.709703 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:58.717273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de1cf6-dc5b-44ec-a155-b0a04bc4d902 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:58.751509 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6f102-0ac6-45cf-9b03-6d4faee6c289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:58.752158 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6f102-0ac6-45cf-9b03-6d4faee6c289 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:58.752438 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6f102-0ac6-45cf-9b03-6d4faee6c289 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:58.752709 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6f102-0ac6-45cf-9b03-6d4faee6c289 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:58.821922 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6f102-0ac6-45cf-9b03-6d4faee6c289 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:58.823420 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6f102-0ac6-45cf-9b03-6d4faee6c289 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:58.925328 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:58.936465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 487/963] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:26:58 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:26:58.942599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f54cf-e6c4-4c48-8490-8257512eaacf None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:58.995689 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:59.050362 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f54cf-e6c4-4c48-8490-8257512eaacf None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:59.086288 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c186a-39c3-4fbf-8679-4ba265b33bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:59.087094 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c186a-39c3-4fbf-8679-4ba265b33bde None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:59.087375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c186a-39c3-4fbf-8679-4ba265b33bde None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:59.087650 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c186a-39c3-4fbf-8679-4ba265b33bde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:59.087755 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a7e47-366f-4906-8f5a-872b0a4222e4 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:59.087928 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a7e47-366f-4906-8f5a-872b0a4222e4 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:59.088211 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a7e47-366f-4906-8f5a-872b0a4222e4 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:59.091388 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 488/964] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:26:59.095700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fff25d-2dad-4f84-8250-774e3ade7e9c None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:59.095838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fff25d-2dad-4f84-8250-774e3ade7e9c None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:59.096035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fff25d-2dad-4f84-8250-774e3ade7e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:59.099996 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c186a-39c3-4fbf-8679-4ba265b33bde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:59.101673 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c186a-39c3-4fbf-8679-4ba265b33bde None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:59.116092 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-39fff25d-2dad-4f84-8250-774e3ade7e9c None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:59.117332 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-39fff25d-2dad-4f84-8250-774e3ade7e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:59.122813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-a46c186a-39c3-4fbf-8679-4ba265b33bde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:59.122813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:59.122813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:59.122813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:59.122813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:59.122813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:59.122813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:59.122813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:59.122813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:59.122813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:59.122813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:59.122813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:59.122813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:26:59.122813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:26:59.122813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:26:59.123905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:26:59.123905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:59.123905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:59.123905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:59.123905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:59.123905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:59.123905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:59.123905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:59.123905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:59.123905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:59.123905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:59.123905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:59.123905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:59.123905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:59.123905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:59.123905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:59.123905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:59.125043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:26:59.125043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:26:59.125043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:26:59.125043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:59.126943 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 477/965] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:26:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:59.137117 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c186a-39c3-4fbf-8679-4ba265b33bde None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:59.194857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 489/966] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:26:59.196191 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3403ee-1eb3-4dea-b9ff-c428edd145f6 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:59.196479 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3403ee-1eb3-4dea-b9ff-c428edd145f6 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:59.196754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3403ee-1eb3-4dea-b9ff-c428edd145f6 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:59.198222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 490/967] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:26:59.204096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa904f-f599-4cca-9002-282e49347d7f None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:59.204381 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa904f-f599-4cca-9002-282e49347d7f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:59.204727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa904f-f599-4cca-9002-282e49347d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:59.223133 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-3bfa904f-f599-4cca-9002-282e49347d7f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:26:59.224880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3bfa904f-f599-4cca-9002-282e49347d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:26:59.240318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:59.244966 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c186a-39c3-4fbf-8679-4ba265b33bde None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:59.283130 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d788cc-45fc-46a0-ba5c-a33335141aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:59.284037 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d788cc-45fc-46a0-ba5c-a33335141aed None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:59.284498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d788cc-45fc-46a0-ba5c-a33335141aed None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:59.284771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d788cc-45fc-46a0-ba5c-a33335141aed None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:59.285522 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d788cc-45fc-46a0-ba5c-a33335141aed None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:59.285826 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d788cc-45fc-46a0-ba5c-a33335141aed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:59.287226 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d788cc-45fc-46a0-ba5c-a33335141aed None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:59.295773 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 491/968] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:26:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:26:59.296896 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa904f-f599-4cca-9002-282e49347d7f None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:59.309835 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 478/969] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:26:59.311120 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d788cc-45fc-46a0-ba5c-a33335141aed None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:59.360243 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:59.408464 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa904f-f599-4cca-9002-282e49347d7f None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:59.436430 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d788cc-45fc-46a0-ba5c-a33335141aed None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:59.445572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2ac06-3f9d-45ec-9fc8-186e526c41be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:59.446300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2ac06-3f9d-45ec-9fc8-186e526c41be None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:59.446587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2ac06-3f9d-45ec-9fc8-186e526c41be None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:59.446853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2ac06-3f9d-45ec-9fc8-186e526c41be None admin] PATH_INFO: `/v3/services/object-store` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:59.447508 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2ac06-3f9d-45ec-9fc8-186e526c41be None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:59.448892 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2ac06-3f9d-45ec-9fc8-186e526c41be None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:59.466489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-24b2ac06-3f9d-45ec-9fc8-186e526c41be None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:26:59.466489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:59.466489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:26:59.466489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:26:59.466489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:26:59.466489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:26:59.466489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:26:59.466489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:59.466489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:26:59.466489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:59.466489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:59.466489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:59.466489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:59.466489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:59.466489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:59.466489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:59.466489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:59.467517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:59.467517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:59.467517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:59.467517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:59.467517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:59.467517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:59.467517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:26:59.467517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:26:59.467517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:26:59.467517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:26:59.467517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:59.467517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:59.467517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:59.467517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:59.467517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:59.467517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:59.468365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:59.468365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:59.468365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:59.468365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:59.468365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:59.468365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:59.468365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:59.468365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:59.468365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:59.468365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:59.468365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:26:59.468365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:26:59.468365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:26:59.468365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:59.468365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 492/970] 10.222.0.71 () {56 vars in 1126 bytes} [Thu Jul 21 20:26:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:59.469230 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2ac06-3f9d-45ec-9fc8-186e526c41be None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:59.478432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885921e-a273-406c-a7a9-4b7eee6e1b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:59.479237 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885921e-a273-406c-a7a9-4b7eee6e1b21 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:59.480067 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885921e-a273-406c-a7a9-4b7eee6e1b21 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:59.480465 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885921e-a273-406c-a7a9-4b7eee6e1b21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:59.488188 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885921e-a273-406c-a7a9-4b7eee6e1b21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:59.489892 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885921e-a273-406c-a7a9-4b7eee6e1b21 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:59.507585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-8885921e-a273-406c-a7a9-4b7eee6e1b21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:59.507585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:59.507585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:59.507585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:59.507585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:59.507585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:59.507585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:59.507585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:59.507585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:59.507585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:59.507585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:59.507585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:59.507585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:26:59.507585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:26:59.507585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:26:59.508516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:26:59.508516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:59.508516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:59.508516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:59.508516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:59.508516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:59.508516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:59.508516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:59.508516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:59.508516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:59.508516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:59.508516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:59.508516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:59.508516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:59.508516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:59.508516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:59.508516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:59.509791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:26:59.509791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:26:59.509791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:26:59.509791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:26:59.509791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:26:59.509791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:26:59.509791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:26:59.509791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:59.509791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 479/971] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:26:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:59.511880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885921e-a273-406c-a7a9-4b7eee6e1b21 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:59.531398 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:59.585421 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2ac06-3f9d-45ec-9fc8-186e526c41be None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:59.609521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:59.620454 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885921e-a273-406c-a7a9-4b7eee6e1b21 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:59.626665 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b730105-d7e4-4f97-939a-f616d7039141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:59.627345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b730105-d7e4-4f97-939a-f616d7039141 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:59.628043 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b730105-d7e4-4f97-939a-f616d7039141 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:59.628337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b730105-d7e4-4f97-939a-f616d7039141 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:59.629041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b730105-d7e4-4f97-939a-f616d7039141 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:59.630877 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b730105-d7e4-4f97-939a-f616d7039141 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:59.632529 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: WARNING keystone.server.flask.common [None req-0b730105-d7e4-4f97-939a-f616d7039141 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 21 20:26:59.645716 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.sql.core [None req-0b730105-d7e4-4f97-939a-f616d7039141 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 20:26:59.645716 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 20:26:59.645716 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [parameters: {'user_id': '9237dba2131548f5b03b736fb51638b0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 20:26:59.645716 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=103663) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 20:26:59.656546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd422ba-0d49-430b-a5b2-81016e5ae04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:59.657311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd422ba-0d49-430b-a5b2-81016e5ae04e None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:59.657593 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd422ba-0d49-430b-a5b2-81016e5ae04e None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:59.657865 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd422ba-0d49-430b-a5b2-81016e5ae04e None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:59.658576 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd422ba-0d49-430b-a5b2-81016e5ae04e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:59.658883 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd422ba-0d49-430b-a5b2-81016e5ae04e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:59.660236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd422ba-0d49-430b-a5b2-81016e5ae04e None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:59.662690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-0b730105-d7e4-4f97-939a-f616d7039141 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 20:26:59.662690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:59.662690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 20:26:59.662690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 20:26:59.662690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 20:26:59.662690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 20:26:59.662690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 20:26:59.662690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 20:26:59.662690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 20:26:59.662690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application conn.query(q) Jul 21 20:26:59.662690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 20:26:59.662690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 20:26:59.662690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 20:26:59.662690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application result.read() Jul 21 20:26:59.662690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 20:26:59.663598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 20:26:59.663598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 20:26:59.663598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 20:26:59.663598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 20:26:59.663598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 20:26:59.663598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 20:26:59.663598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 20:26:59.663598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 20:26:59.663598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:59.663598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 20:26:59.663598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:59.663598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:59.663598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 20:26:59.663598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 20:26:59.663598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 21 20:26:59.663598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 21 20:26:59.663598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 20:26:59.663598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application next(self.gen) Jul 21 20:26:59.664597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 20:26:59.664597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application yield resource Jul 21 20:26:59.664597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 20:26:59.664597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application next(self.gen) Jul 21 20:26:59.664597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 20:26:59.664597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 20:26:59.664597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 20:26:59.664597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 20:26:59.664597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 20:26:59.664597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise self.value Jul 21 20:26:59.664597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 20:26:59.664597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 20:26:59.664597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 20:26:59.664597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application session.commit() Jul 21 20:26:59.664597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 20:26:59.664597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 20:26:59.664597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 20:26:59.666392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 20:26:59.666392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 20:26:59.666392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self.session.flush() Jul 21 20:26:59.666392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 20:26:59.666392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 20:26:59.666392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 20:26:59.666392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 20:26:59.666392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 20:26:59.666392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application compat.raise_( Jul 21 20:26:59.666392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 20:26:59.666392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception Jul 21 20:26:59.666392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 20:26:59.666392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 20:26:59.666392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 20:26:59.666392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 20:26:59.666392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 20:26:59.666392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 20:26:59.667766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 20:26:59.667766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 20:26:59.667766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 21 20:26:59.667766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 21 20:26:59.667766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 20:26:59.667766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 20:26:59.667766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 20:26:59.667766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 20:26:59.667766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 20:26:59.667766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 20:26:59.667766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 20:26:59.667766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 20:26:59.667766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 20:26:59.667766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 20:26:59.667766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 20:26:59.667766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception Jul 21 20:26:59.670572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 20:26:59.670572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 20:26:59.670572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 20:26:59.670572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 20:26:59.670572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 20:26:59.670572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 20:26:59.670572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 20:26:59.670572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application conn.query(q) Jul 21 20:26:59.670572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 20:26:59.670572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 20:26:59.670572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 20:26:59.670572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application result.read() Jul 21 20:26:59.670572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 20:26:59.670572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 20:26:59.670572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 20:26:59.670572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 20:26:59.672603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 20:26:59.672603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 20:26:59.672603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 20:26:59.672603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 20:26:59.672603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 20:26:59.672603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 20:26:59.672603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [parameters: {'user_id': '9237dba2131548f5b03b736fb51638b0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 20:26:59.672603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 20:26:59.672603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:59.672603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:26:59.672603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:59.672603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:59.672603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:59.672603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:59.672603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:59.673662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:59.673662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:59.673662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:59.673662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:59.673662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:59.673662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:59.673662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:59.673662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 21 20:26:59.673662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 21 20:26:59.673662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:59.673662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:59.673662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:59.673662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:59.673662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:59.673662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:59.673662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 21 20:26:59.673662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 21 20:26:59.674963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 21 20:26:59.674963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 21 20:26:59.674963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 20:26:59.674963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 20:26:59.674963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 20:26:59.674963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:26:59.674963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 493/972] 10.222.0.71 () {60 vars in 1131 bytes} [Thu Jul 21 20:26:59 2022] POST /identity/v3/users => generated 160 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:59.674963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b730105-d7e4-4f97-939a-f616d7039141 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:59.675628 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 480/973] 10.222.0.71 () {56 vars in 1122 bytes} [Thu Jul 21 20:26:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:59.676499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd422ba-0d49-430b-a5b2-81016e5ae04e None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:59.772338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:59.786737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b730105-d7e4-4f97-939a-f616d7039141 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:59.792105 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd422ba-0d49-430b-a5b2-81016e5ae04e None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:59.824531 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e769d9-31f6-4b5d-a6d9-cecc1bdcb1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:59.825213 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e769d9-31f6-4b5d-a6d9-cecc1bdcb1f2 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:59.825496 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e769d9-31f6-4b5d-a6d9-cecc1bdcb1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:59.825767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e769d9-31f6-4b5d-a6d9-cecc1bdcb1f2 None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:59.826546 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e769d9-31f6-4b5d-a6d9-cecc1bdcb1f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:26:59.826929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e769d9-31f6-4b5d-a6d9-cecc1bdcb1f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:59.828557 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e769d9-31f6-4b5d-a6d9-cecc1bdcb1f2 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:59.829022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d34dd-44dc-46c6-9fef-ad215ce853d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:26:59.829846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d34dd-44dc-46c6-9fef-ad215ce853d8 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:26:59.830312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d34dd-44dc-46c6-9fef-ad215ce853d8 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:26:59.830592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d34dd-44dc-46c6-9fef-ad215ce853d8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:26:59.839484 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d34dd-44dc-46c6-9fef-ad215ce853d8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:26:59.841104 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d34dd-44dc-46c6-9fef-ad215ce853d8 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:26:59.849974 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 494/974] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:26:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:26:59.849974 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e769d9-31f6-4b5d-a6d9-cecc1bdcb1f2 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:59.858793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-fd5d34dd-44dc-46c6-9fef-ad215ce853d8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 20:26:59.858793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:26:59.858793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:26:59.858793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:26:59.858793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:26:59.858793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:26:59.858793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:26:59.858793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:26:59.858793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:26:59.858793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:26:59.858793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:26:59.858793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:26:59.858793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:26:59.858793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:26:59.858793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:26:59.858793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:26:59.859937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:26:59.859937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:26:59.859937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:26:59.859937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:59.859937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:26:59.859937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:26:59.859937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:26:59.859937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:26:59.859937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:26:59.859937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:26:59.859937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:26:59.859937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:26:59.859937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:26:59.859937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:26:59.859937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:26:59.859937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:26:59.859937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:26:59.859937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:26:59.860984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:26:59.860984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:26:59.860984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:26:59.860984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:26:59.860984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:26:59.860984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:26:59.860984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:26:59.860984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:26:59.860984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 20:26:59.860984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:26:59.860984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 481/975] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:26:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:26:59.861644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d34dd-44dc-46c6-9fef-ad215ce853d8 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:26:59.903144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:26:59.976689 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d34dd-44dc-46c6-9fef-ad215ce853d8 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:26:59.978231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e769d9-31f6-4b5d-a6d9-cecc1bdcb1f2 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:00.014336 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f052c-db00-425b-afd2-3c364623a2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:00.015152 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f052c-db00-425b-afd2-3c364623a2c6 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:00.015535 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f052c-db00-425b-afd2-3c364623a2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:00.015909 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f052c-db00-425b-afd2-3c364623a2c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:00.016505 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b4c70-5392-4ad5-8428-4f2bbd8220dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:00.016950 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f052c-db00-425b-afd2-3c364623a2c6 None admin] RBAC: Adding query filter params (role.id=4eefa64717a84653822b16532ebf6e75, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=b94e634e8855486a9ce14d3de9d986fb) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:00.017183 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b4c70-5392-4ad5-8428-4f2bbd8220dd None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:00.017468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b4c70-5392-4ad5-8428-4f2bbd8220dd None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:00.017559 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f052c-db00-425b-afd2-3c364623a2c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:00.017738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b4c70-5392-4ad5-8428-4f2bbd8220dd None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:00.018438 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b4c70-5392-4ad5-8428-4f2bbd8220dd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:00.018738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b4c70-5392-4ad5-8428-4f2bbd8220dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:00.019343 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f052c-db00-425b-afd2-3c364623a2c6 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:00.020073 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b4c70-5392-4ad5-8428-4f2bbd8220dd None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:00.033410 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 495/976] 10.222.0.71 () {56 vars in 1122 bytes} [Thu Jul 21 20:26:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:00.033585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 482/977] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:26:59 2022] GET /identity/v3/role_assignments?user.id=b94e634e8855486a9ce14d3de9d986fb&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=4eefa64717a84653822b16532ebf6e75 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:00.033703 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b4c70-5392-4ad5-8428-4f2bbd8220dd None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:00.033802 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f052c-db00-425b-afd2-3c364623a2c6 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:00.043526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:00.155170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b4c70-5392-4ad5-8428-4f2bbd8220dd None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:00.163804 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:00.171455 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f052c-db00-425b-afd2-3c364623a2c6 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:00.198118 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235934ad-a23c-48e9-a03a-36e921b6cb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:00.200187 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235934ad-a23c-48e9-a03a-36e921b6cb75 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:00.200984 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235934ad-a23c-48e9-a03a-36e921b6cb75 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:00.201381 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235934ad-a23c-48e9-a03a-36e921b6cb75 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:00.202321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235934ad-a23c-48e9-a03a-36e921b6cb75 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:00.203952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235934ad-a23c-48e9-a03a-36e921b6cb75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:00.206013 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235934ad-a23c-48e9-a03a-36e921b6cb75 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:00.215770 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc35bf-e108-4b43-82c5-63a097f119f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:00.216493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc35bf-e108-4b43-82c5-63a097f119f5 None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:00.216784 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc35bf-e108-4b43-82c5-63a097f119f5 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:00.217110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc35bf-e108-4b43-82c5-63a097f119f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:00.217882 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc35bf-e108-4b43-82c5-63a097f119f5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:00.221000 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc35bf-e108-4b43-82c5-63a097f119f5 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:00.246882 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 483/978] 10.222.0.71 () {60 vars in 1144 bytes} [Thu Jul 21 20:27:00 2022] POST /identity/v3/endpoints => generated 349 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:27:00.257203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 496/979] 10.222.0.71 () {56 vars in 1101 bytes} [Thu Jul 21 20:27:00 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:00.690513 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5424ac-caf9-41e0-bd09-2dbe5beb5910 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:00.690880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5424ac-caf9-41e0-bd09-2dbe5beb5910 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:00.691053 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5424ac-caf9-41e0-bd09-2dbe5beb5910 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:00.692472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 484/980] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:00.697700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299145a-8fa1-432e-8587-6119a23b771d None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:00.698174 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299145a-8fa1-432e-8587-6119a23b771d None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:00.698353 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299145a-8fa1-432e-8587-6119a23b771d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:00.715891 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-2299145a-8fa1-432e-8587-6119a23b771d None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:00.717343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-2299145a-8fa1-432e-8587-6119a23b771d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:00.804454 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 497/981] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:00.827883 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5424ac-caf9-41e0-bd09-2dbe5beb5910 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:00.892956 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:00.936492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5424ac-caf9-41e0-bd09-2dbe5beb5910 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:00.972820 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e59ec-74c0-47c6-a58f-21bf9f27d0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:00.973770 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e59ec-74c0-47c6-a58f-21bf9f27d0f0 None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:00.974116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e59ec-74c0-47c6-a58f-21bf9f27d0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:00.974455 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e59ec-74c0-47c6-a58f-21bf9f27d0f0 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:00.975309 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e59ec-74c0-47c6-a58f-21bf9f27d0f0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:00.976782 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e59ec-74c0-47c6-a58f-21bf9f27d0f0 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:00.992433 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 485/982] 10.222.0.71 () {60 vars in 1141 bytes} [Thu Jul 21 20:27:00 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:27:02.192635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ceccba-dd31-4e8b-b4b9-a67200fc5ced None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:02.193397 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ceccba-dd31-4e8b-b4b9-a67200fc5ced None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:02.193397 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ceccba-dd31-4e8b-b4b9-a67200fc5ced None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:02.194745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 498/983] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:02.200520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98b83c-a670-43bd-a5e6-b00a560d62d4 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:02.201056 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98b83c-a670-43bd-a5e6-b00a560d62d4 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:02.201056 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98b83c-a670-43bd-a5e6-b00a560d62d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:02.217829 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-4c98b83c-a670-43bd-a5e6-b00a560d62d4 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:02.219016 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-4c98b83c-a670-43bd-a5e6-b00a560d62d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:02.295789 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 486/984] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:02 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:02.317541 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ceccba-dd31-4e8b-b4b9-a67200fc5ced None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:02.357930 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a362fff-0ae6-4206-b836-06164f7390b5 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:02.358241 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a362fff-0ae6-4206-b836-06164f7390b5 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:02.358503 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a362fff-0ae6-4206-b836-06164f7390b5 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:02.360030 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 487/985] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:27:02.365044 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa1a71-1473-4ba9-9f61-70ca0efd767c None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:02.365348 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa1a71-1473-4ba9-9f61-70ca0efd767c None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:02.365638 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa1a71-1473-4ba9-9f61-70ca0efd767c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:02.382882 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-9baa1a71-1473-4ba9-9f61-70ca0efd767c None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:02.384272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-9baa1a71-1473-4ba9-9f61-70ca0efd767c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:02.418567 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:02.430081 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ceccba-dd31-4e8b-b4b9-a67200fc5ced None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:02.431920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:02.461716 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 488/986] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:02 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:27:02.467604 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3907c4-574c-43a7-b0b5-78adc13b02c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:02.468598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3907c4-574c-43a7-b0b5-78adc13b02c9 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:02.469043 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3907c4-574c-43a7-b0b5-78adc13b02c9 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:02.469498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3907c4-574c-43a7-b0b5-78adc13b02c9 None admin] PATH_INFO: `/v3/services/image` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:02.470296 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3907c4-574c-43a7-b0b5-78adc13b02c9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:02.471812 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3907c4-574c-43a7-b0b5-78adc13b02c9 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:02.485904 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa1a71-1473-4ba9-9f61-70ca0efd767c None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:02.486779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-2b3907c4-574c-43a7-b0b5-78adc13b02c9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 20:27:02.486779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:02.486779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:27:02.486779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:27:02.486779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:27:02.486779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:27:02.486779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:27:02.486779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:27:02.486779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 20:27:02.486779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:02.486779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:27:02.486779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:02.486779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:02.486779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:02.486779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:02.486779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:02.486779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:02.487952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:02.487952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:02.487952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:02.487952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:02.487952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:02.487952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:02.487952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:27:02.487952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:27:02.487952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:27:02.487952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:27:02.487952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:02.487952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:02.487952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:02.487952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:02.487952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:02.487952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:02.489003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:02.489003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:02.489003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:02.489003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:02.489003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:02.489003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:02.489003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:02.489003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:02.489003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:02.489003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:02.489003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:27:02.489003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:27:02.489003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 20:27:02.489003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:02.491675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 499/987] 10.222.0.71 () {56 vars in 1105 bytes} [Thu Jul 21 20:27:02 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:02.497598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3907c4-574c-43a7-b0b5-78adc13b02c9 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:02.600460 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa1a71-1473-4ba9-9f61-70ca0efd767c None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:02.616293 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3907c4-574c-43a7-b0b5-78adc13b02c9 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:02.636586 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a276421-8a59-4601-880e-98959da7b51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:02.637424 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a276421-8a59-4601-880e-98959da7b51e None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:02.637707 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a276421-8a59-4601-880e-98959da7b51e None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:02.637986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a276421-8a59-4601-880e-98959da7b51e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:02.638654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a276421-8a59-4601-880e-98959da7b51e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:02.640018 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a276421-8a59-4601-880e-98959da7b51e None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:02.652220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-5a276421-8a59-4601-880e-98959da7b51e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:27:02.652220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:02.652220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:27:02.652220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:27:02.652220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:27:02.652220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:27:02.652220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:27:02.652220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:27:02.652220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:27:02.652220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:02.652220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:27:02.652220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:02.652220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:02.652220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:02.652220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:02.652220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:02.652220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:02.653281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:02.653281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:02.653281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:02.653281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:02.653281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:02.653281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:02.653281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:27:02.653281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:27:02.653281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:27:02.653281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:27:02.653281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:02.653281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:02.653281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:02.653281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:02.653281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:02.653281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:02.658070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:02.658070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:02.658070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:02.658070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:02.658070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:02.658070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:02.658070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:02.658070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:02.658070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:02.658070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:02.658070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:27:02.658070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:27:02.658070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:27:02.658070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:02.658823 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c50945-6bbe-444c-a48f-9a772d3cf41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:02.658823 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c50945-6bbe-444c-a48f-9a772d3cf41b None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:02.658823 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c50945-6bbe-444c-a48f-9a772d3cf41b None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:02.659045 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 489/988] 10.222.0.71 () {56 vars in 1126 bytes} [Thu Jul 21 20:27:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:27:02.659116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c50945-6bbe-444c-a48f-9a772d3cf41b None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:02.659116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c50945-6bbe-444c-a48f-9a772d3cf41b None admin] RBAC: Adding query filter params (name=image) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:02.659116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c50945-6bbe-444c-a48f-9a772d3cf41b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:02.659116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c50945-6bbe-444c-a48f-9a772d3cf41b None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:02.660472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a276421-8a59-4601-880e-98959da7b51e None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:02.661743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 500/989] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:27:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:02.667059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c50945-6bbe-444c-a48f-9a772d3cf41b None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:02.687439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:02.707295 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:02.771170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a276421-8a59-4601-880e-98959da7b51e None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:02.777609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c50945-6bbe-444c-a48f-9a772d3cf41b None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:02.806130 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480581b-9b92-44d9-b54b-ec6283fcb69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:02.806793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480581b-9b92-44d9-b54b-ec6283fcb69b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:02.807079 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480581b-9b92-44d9-b54b-ec6283fcb69b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:02.807353 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480581b-9b92-44d9-b54b-ec6283fcb69b None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:02.808594 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480581b-9b92-44d9-b54b-ec6283fcb69b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:02.808906 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480581b-9b92-44d9-b54b-ec6283fcb69b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:02.810294 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480581b-9b92-44d9-b54b-ec6283fcb69b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:02.813957 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d8922-95f2-4ee0-851c-4b9719484a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:02.814616 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d8922-95f2-4ee0-851c-4b9719484a6d None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:02.814923 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d8922-95f2-4ee0-851c-4b9719484a6d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:02.815173 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d8922-95f2-4ee0-851c-4b9719484a6d None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:02.820100 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 490/990] 10.222.0.71 () {56 vars in 1122 bytes} [Thu Jul 21 20:27:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:02.820255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d8922-95f2-4ee0-851c-4b9719484a6d None admin] RBAC: Adding query filter params (type=image) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:02.820255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d8922-95f2-4ee0-851c-4b9719484a6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:02.820255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d8922-95f2-4ee0-851c-4b9719484a6d None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:02.821041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480581b-9b92-44d9-b54b-ec6283fcb69b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:02.824701 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 501/991] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:27:02 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:02.943158 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480581b-9b92-44d9-b54b-ec6283fcb69b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:02.953709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:02.988243 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34064e9c-4804-48e2-b437-7b50c9879ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:02.989995 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34064e9c-4804-48e2-b437-7b50c9879ec6 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:02.990466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34064e9c-4804-48e2-b437-7b50c9879ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:02.990798 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34064e9c-4804-48e2-b437-7b50c9879ec6 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:02.991951 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34064e9c-4804-48e2-b437-7b50c9879ec6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:02.992165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34064e9c-4804-48e2-b437-7b50c9879ec6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:02.994163 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34064e9c-4804-48e2-b437-7b50c9879ec6 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:03.000211 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 491/992] 10.222.0.71 () {56 vars in 1122 bytes} [Thu Jul 21 20:27:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:03.006459 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d8922-95f2-4ee0-851c-4b9719484a6d None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:03.037106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da56ee-99ee-48c1-9d77-db1eafaf177f None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:03.037383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da56ee-99ee-48c1-9d77-db1eafaf177f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:03.037663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da56ee-99ee-48c1-9d77-db1eafaf177f None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:03.039198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 492/993] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:27:03.044578 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4ae41-dc90-4850-9922-2a666eabbb6f None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:03.044864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4ae41-dc90-4850-9922-2a666eabbb6f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:03.045171 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4ae41-dc90-4850-9922-2a666eabbb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:03.061983 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-26d4ae41-dc90-4850-9922-2a666eabbb6f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:03.063212 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-26d4ae41-dc90-4850-9922-2a666eabbb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:03.098064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:03.118997 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d8922-95f2-4ee0-851c-4b9719484a6d None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:03.138683 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 493/994] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:27:03.156530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a0294-df40-4825-9f29-34ca44a73d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:03.157221 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a0294-df40-4825-9f29-34ca44a73d57 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:03.157506 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a0294-df40-4825-9f29-34ca44a73d57 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:03.157781 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a0294-df40-4825-9f29-34ca44a73d57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:03.158584 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a0294-df40-4825-9f29-34ca44a73d57 None admin] RBAC: Adding query filter params (interface=admin, service_id=92e0d2d675ed48d1811a544e7d184728, region_id=RegionOne) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:03.158899 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a0294-df40-4825-9f29-34ca44a73d57 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:03.160277 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a0294-df40-4825-9f29-34ca44a73d57 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:03.163532 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4ae41-dc90-4850-9922-2a666eabbb6f None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:03.171395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 502/995] 10.222.0.71 () {56 vars in 1249 bytes} [Thu Jul 21 20:27:03 2022] GET /identity/v3/endpoints?service_id=92e0d2d675ed48d1811a544e7d184728&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:03.172356 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a0294-df40-4825-9f29-34ca44a73d57 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:03.270437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:03.274651 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4ae41-dc90-4850-9922-2a666eabbb6f None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:03.280340 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a0294-df40-4825-9f29-34ca44a73d57 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:03.315499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba240e6-e4e6-4080-bc2b-d580e3d750d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:03.316297 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba240e6-e4e6-4080-bc2b-d580e3d750d6 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:03.316543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13407c-eb69-4f40-8b4e-b3935c52f615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:03.316672 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba240e6-e4e6-4080-bc2b-d580e3d750d6 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:03.316849 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba240e6-e4e6-4080-bc2b-d580e3d750d6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:03.317245 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13407c-eb69-4f40-8b4e-b3935c52f615 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:03.317525 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13407c-eb69-4f40-8b4e-b3935c52f615 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:03.317626 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba240e6-e4e6-4080-bc2b-d580e3d750d6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:03.317846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13407c-eb69-4f40-8b4e-b3935c52f615 None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:03.318496 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13407c-eb69-4f40-8b4e-b3935c52f615 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:03.318880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba240e6-e4e6-4080-bc2b-d580e3d750d6 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:03.319854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13407c-eb69-4f40-8b4e-b3935c52f615 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:03.327712 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 503/996] 10.222.0.71 () {56 vars in 1087 bytes} [Thu Jul 21 20:27:03 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:27:03.331275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-fba240e6-e4e6-4080-bc2b-d580e3d750d6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 20:27:03.331275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:03.331275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:27:03.331275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:27:03.331275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:27:03.331275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:27:03.331275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:27:03.331275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:27:03.331275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 20:27:03.331275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:03.331275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:27:03.331275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:03.331275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:03.331275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:03.331275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:03.331275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:03.331275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:03.332479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:03.332479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:03.332479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:03.332479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:03.332479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:03.332479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:03.332479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:27:03.332479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:27:03.332479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:27:03.332479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:27:03.332479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:03.332479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:03.332479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:03.332479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:03.332479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:03.332479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:03.333822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:03.333822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:03.333822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:03.333822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:03.333822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:03.333822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:03.333822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:03.333822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:03.333822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:03.333822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:03.333822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:27:03.333822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:27:03.333822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 20:27:03.333822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:03.337625 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 494/997] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:27:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:27:03.338984 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13407c-eb69-4f40-8b4e-b3935c52f615 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:03.353493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:03.455408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13407c-eb69-4f40-8b4e-b3935c52f615 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:03.499873 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae6749-7074-4432-92c3-4a1f183d209a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:03.500554 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae6749-7074-4432-92c3-4a1f183d209a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:03.500832 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae6749-7074-4432-92c3-4a1f183d209a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:03.501131 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae6749-7074-4432-92c3-4a1f183d209a None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:03.501873 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae6749-7074-4432-92c3-4a1f183d209a None admin] RBAC: Adding query filter params (name=compute) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:03.502184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae6749-7074-4432-92c3-4a1f183d209a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:03.503558 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae6749-7074-4432-92c3-4a1f183d209a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:03.509169 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 504/998] 10.222.0.71 () {56 vars in 1112 bytes} [Thu Jul 21 20:27:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:03.515183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba240e6-e4e6-4080-bc2b-d580e3d750d6 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:03.628053 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba240e6-e4e6-4080-bc2b-d580e3d750d6 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:03.667982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216171d-4339-489c-95ec-493e7a534c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:03.668826 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216171d-4339-489c-95ec-493e7a534c69 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:03.669272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216171d-4339-489c-95ec-493e7a534c69 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:03.669556 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216171d-4339-489c-95ec-493e7a534c69 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:03.670290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216171d-4339-489c-95ec-493e7a534c69 None admin] RBAC: Adding query filter params (type=compute) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:03.670740 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216171d-4339-489c-95ec-493e7a534c69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:03.672361 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216171d-4339-489c-95ec-493e7a534c69 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:03.683553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 495/999] 10.222.0.71 () {56 vars in 1112 bytes} [Thu Jul 21 20:27:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:03.685206 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae6749-7074-4432-92c3-4a1f183d209a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:03.773398 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:03.808491 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae6749-7074-4432-92c3-4a1f183d209a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:03.853315 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4f5bc-0e59-4e72-a720-b65f4079d4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:03.854011 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4f5bc-0e59-4e72-a720-b65f4079d4b8 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:03.854429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4f5bc-0e59-4e72-a720-b65f4079d4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:03.854774 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4f5bc-0e59-4e72-a720-b65f4079d4b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:03.855686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4f5bc-0e59-4e72-a720-b65f4079d4b8 None admin] RBAC: Adding query filter params (interface=public, service_id=fced55d53e4d40d3868551bccfa325fa, region_id=RegionOne) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:03.856020 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4f5bc-0e59-4e72-a720-b65f4079d4b8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:03.857426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4f5bc-0e59-4e72-a720-b65f4079d4b8 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:03.867206 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 505/1000] 10.222.0.71 () {56 vars in 1251 bytes} [Thu Jul 21 20:27:03 2022] GET /identity/v3/endpoints?service_id=fced55d53e4d40d3868551bccfa325fa&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:03.868778 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216171d-4339-489c-95ec-493e7a534c69 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:03.899832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:03.980556 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216171d-4339-489c-95ec-493e7a534c69 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:04.017039 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f529c2-5cd2-468e-838e-fa62f3198a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:04.017739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f529c2-5cd2-468e-838e-fa62f3198a95 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:04.018015 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f529c2-5cd2-468e-838e-fa62f3198a95 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:04.018442 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f529c2-5cd2-468e-838e-fa62f3198a95 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:04.019744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f529c2-5cd2-468e-838e-fa62f3198a95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:04.021301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f529c2-5cd2-468e-838e-fa62f3198a95 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:04.037142 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 496/1001] 10.222.0.71 () {56 vars in 1087 bytes} [Thu Jul 21 20:27:03 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:27:04.913027 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042b231-8ea6-44c6-9470-64c67d923787 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:04.913579 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042b231-8ea6-44c6-9470-64c67d923787 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:04.913686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042b231-8ea6-44c6-9470-64c67d923787 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:04.915040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 506/1002] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:04.920219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a4fe4-f185-49d0-bcc0-dea6bb5d842c None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:04.920621 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a4fe4-f185-49d0-bcc0-dea6bb5d842c None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:04.920784 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a4fe4-f185-49d0-bcc0-dea6bb5d842c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:04.938087 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-a17a4fe4-f185-49d0-bcc0-dea6bb5d842c None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:04.939283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-a17a4fe4-f185-49d0-bcc0-dea6bb5d842c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:05.011127 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 497/1003] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:04 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:05.033572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042b231-8ea6-44c6-9470-64c67d923787 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:05.138256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042b231-8ea6-44c6-9470-64c67d923787 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:05.175484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86e6da-c83f-4e25-ba48-8c2ca7fea4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:05.176065 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86e6da-c83f-4e25-ba48-8c2ca7fea4ed None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:05.176345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86e6da-c83f-4e25-ba48-8c2ca7fea4ed None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:05.176619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86e6da-c83f-4e25-ba48-8c2ca7fea4ed None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:05.177406 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a86e6da-c83f-4e25-ba48-8c2ca7fea4ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:05.178789 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a86e6da-c83f-4e25-ba48-8c2ca7fea4ed None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:05.187137 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:05.195997 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 507/1004] 10.222.0.71 () {60 vars in 1141 bytes} [Thu Jul 21 20:27:05 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:27:05.398740 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea3435-b1b3-42bc-a44b-85889b4df32f None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:05.399128 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea3435-b1b3-42bc-a44b-85889b4df32f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:05.399325 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea3435-b1b3-42bc-a44b-85889b4df32f None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:05.400662 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 498/1005] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:05.405920 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff0244-99b4-453c-b74f-bd0e357a3140 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:05.406209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff0244-99b4-453c-b74f-bd0e357a3140 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:05.406481 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff0244-99b4-453c-b74f-bd0e357a3140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:05.423487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-3cff0244-99b4-453c-b74f-bd0e357a3140 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:05.424658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3cff0244-99b4-453c-b74f-bd0e357a3140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:05.511449 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 508/1006] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:05 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:05.536270 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea3435-b1b3-42bc-a44b-85889b4df32f None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:05.563141 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:05.645839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea3435-b1b3-42bc-a44b-85889b4df32f None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:05.681973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54077b9c-920f-4107-ad2b-c18a22b4d9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:05.683327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54077b9c-920f-4107-ad2b-c18a22b4d9be None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:05.683608 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54077b9c-920f-4107-ad2b-c18a22b4d9be None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:05.683879 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54077b9c-920f-4107-ad2b-c18a22b4d9be None admin] PATH_INFO: `/v3/services/object-store` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:05.684532 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54077b9c-920f-4107-ad2b-c18a22b4d9be None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:05.685932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54077b9c-920f-4107-ad2b-c18a22b4d9be None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:05.697982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-54077b9c-920f-4107-ad2b-c18a22b4d9be None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:27:05.697982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:05.697982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:27:05.697982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:27:05.697982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:27:05.697982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:27:05.697982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:27:05.697982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:27:05.697982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:27:05.697982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:05.697982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:27:05.697982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:05.697982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:05.697982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:05.697982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:05.697982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:05.697982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:05.699038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:05.699038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:05.699038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:05.699038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:05.699038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:05.699038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:05.699038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:27:05.699038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:27:05.699038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:27:05.699038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:27:05.699038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:05.699038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:05.699038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:05.699038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:05.699038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:05.699038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:05.705135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:05.705135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:05.705135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:05.705135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:05.705135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:05.705135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:05.705135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:05.705135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:05.705135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:05.705135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:05.705135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:27:05.705135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:27:05.705135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:27:05.705135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:05.705135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 499/1007] 10.222.0.71 () {56 vars in 1126 bytes} [Thu Jul 21 20:27:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:05.707554 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff0244-99b4-453c-b74f-bd0e357a3140 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:05.810244 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff0244-99b4-453c-b74f-bd0e357a3140 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:05.833655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:05.847510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be772813-18e8-4710-a645-5fb874828b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:05.848489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be772813-18e8-4710-a645-5fb874828b82 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:05.848798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be772813-18e8-4710-a645-5fb874828b82 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:05.849140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be772813-18e8-4710-a645-5fb874828b82 None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:05.849861 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be772813-18e8-4710-a645-5fb874828b82 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:05.850171 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be772813-18e8-4710-a645-5fb874828b82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:05.851551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be772813-18e8-4710-a645-5fb874828b82 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:05.862358 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 509/1008] 10.222.0.71 () {56 vars in 1122 bytes} [Thu Jul 21 20:27:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:05.863703 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54077b9c-920f-4107-ad2b-c18a22b4d9be None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:05.952584 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:05.969824 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54077b9c-920f-4107-ad2b-c18a22b4d9be None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:06.003743 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d077a-7930-495b-8df4-3d3cdf156749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:06.004450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d077a-7930-495b-8df4-3d3cdf156749 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:06.004751 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d077a-7930-495b-8df4-3d3cdf156749 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:06.005107 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d077a-7930-495b-8df4-3d3cdf156749 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:06.005820 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d077a-7930-495b-8df4-3d3cdf156749 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:06.006126 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d077a-7930-495b-8df4-3d3cdf156749 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:06.007474 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d077a-7930-495b-8df4-3d3cdf156749 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:06.013418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 500/1009] 10.222.0.71 () {56 vars in 1122 bytes} [Thu Jul 21 20:27:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:06.018904 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be772813-18e8-4710-a645-5fb874828b82 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:06.107902 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a808e-4ecc-4d70-9581-30330395dfae None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:06.108249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a808e-4ecc-4d70-9581-30330395dfae None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:06.108463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a808e-4ecc-4d70-9581-30330395dfae None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:06.109903 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 501/1010] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:27:06.114659 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcbb7b-8d68-41a8-be11-b84d97df52de None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:06.114942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcbb7b-8d68-41a8-be11-b84d97df52de None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:06.115260 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcbb7b-8d68-41a8-be11-b84d97df52de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:06.124457 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be772813-18e8-4710-a645-5fb874828b82 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:06.132260 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-defcbb7b-8d68-41a8-be11-b84d97df52de None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:06.133458 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-defcbb7b-8d68-41a8-be11-b84d97df52de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:06.159013 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695cb23a-259d-4b3a-8766-5d7c4623ec22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:06.159664 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695cb23a-259d-4b3a-8766-5d7c4623ec22 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:06.159942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695cb23a-259d-4b3a-8766-5d7c4623ec22 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:06.160216 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695cb23a-259d-4b3a-8766-5d7c4623ec22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:06.160956 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695cb23a-259d-4b3a-8766-5d7c4623ec22 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:06.162350 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695cb23a-259d-4b3a-8766-5d7c4623ec22 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:06.189113 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 510/1011] 10.222.0.71 () {60 vars in 1144 bytes} [Thu Jul 21 20:27:06 2022] POST /identity/v3/endpoints => generated 325 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:27:06.214656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 502/1012] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:06 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:27:06.238530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695cb23a-259d-4b3a-8766-5d7c4623ec22 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:06.293954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:06.356948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695cb23a-259d-4b3a-8766-5d7c4623ec22 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:06.392834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff0ee9-20a3-4dae-8758-f5814f30a772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:06.394037 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff0ee9-20a3-4dae-8758-f5814f30a772 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:06.394541 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff0ee9-20a3-4dae-8758-f5814f30a772 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:06.395010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff0ee9-20a3-4dae-8758-f5814f30a772 None admin] PATH_INFO: `/v3/services/compute` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:06.395866 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff0ee9-20a3-4dae-8758-f5814f30a772 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:06.397469 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff0ee9-20a3-4dae-8758-f5814f30a772 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:06.410203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-6aff0ee9-20a3-4dae-8758-f5814f30a772 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 20:27:06.410203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:06.410203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:27:06.410203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:27:06.410203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:27:06.410203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:27:06.410203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:27:06.410203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:27:06.410203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 20:27:06.410203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:06.410203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:27:06.410203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:06.410203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:06.410203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:06.410203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:06.410203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:06.410203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:06.411301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:06.411301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:06.411301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:06.411301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:06.411301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:06.411301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:06.411301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:27:06.411301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:27:06.411301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:27:06.411301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:27:06.411301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:06.411301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:06.411301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:06.411301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:06.411301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:06.411301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:06.412176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:06.412176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:06.412176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:06.412176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:06.412176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:06.412176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:06.412176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:06.412176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:06.412176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:06.412176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:06.412176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:27:06.412176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:27:06.412176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 20:27:06.412176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:06.412920 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 511/1013] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:27:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:06.418289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcbb7b-8d68-41a8-be11-b84d97df52de None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:06.501813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:06.526843 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcbb7b-8d68-41a8-be11-b84d97df52de None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:06.561568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2c4d7-b672-4a3b-9486-e07cd1fce45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:06.562233 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2c4d7-b672-4a3b-9486-e07cd1fce45c None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:06.562513 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2c4d7-b672-4a3b-9486-e07cd1fce45c None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:06.562786 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2c4d7-b672-4a3b-9486-e07cd1fce45c None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:06.563496 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2c4d7-b672-4a3b-9486-e07cd1fce45c None admin] RBAC: Adding query filter params (name=compute) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:06.563804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2c4d7-b672-4a3b-9486-e07cd1fce45c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:06.565198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2c4d7-b672-4a3b-9486-e07cd1fce45c None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:06.575419 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 503/1014] 10.222.0.71 () {56 vars in 1112 bytes} [Thu Jul 21 20:27:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:06.576647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff0ee9-20a3-4dae-8758-f5814f30a772 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:06.681035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff0ee9-20a3-4dae-8758-f5814f30a772 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:06.694811 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:06.717019 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e0aba-b045-4d96-929e-12333fd28f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:06.717699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e0aba-b045-4d96-929e-12333fd28f5e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:06.717979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e0aba-b045-4d96-929e-12333fd28f5e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:06.718255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e0aba-b045-4d96-929e-12333fd28f5e None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:06.718956 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e0aba-b045-4d96-929e-12333fd28f5e None admin] RBAC: Adding query filter params (type=compute) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:06.719897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e0aba-b045-4d96-929e-12333fd28f5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:06.720623 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e0aba-b045-4d96-929e-12333fd28f5e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:06.726028 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 512/1015] 10.222.0.71 () {56 vars in 1112 bytes} [Thu Jul 21 20:27:06 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:06.731538 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2c4d7-b672-4a3b-9486-e07cd1fce45c None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:06.833016 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2c4d7-b672-4a3b-9486-e07cd1fce45c None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:06.870817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0288e7-8aff-40df-bd6f-cae6a1400579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:06.871483 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0288e7-8aff-40df-bd6f-cae6a1400579 None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:06.871753 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0288e7-8aff-40df-bd6f-cae6a1400579 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:06.872018 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0288e7-8aff-40df-bd6f-cae6a1400579 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:06.872771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0288e7-8aff-40df-bd6f-cae6a1400579 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:06.874166 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0288e7-8aff-40df-bd6f-cae6a1400579 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:06.882278 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:06.896185 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 504/1016] 10.222.0.71 () {60 vars in 1144 bytes} [Thu Jul 21 20:27:06 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:27:07.280011 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721184ad-09f7-472f-8a79-690799d99c0d None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:07.280537 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721184ad-09f7-472f-8a79-690799d99c0d None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:07.280537 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721184ad-09f7-472f-8a79-690799d99c0d None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:07.282122 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 513/1017] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:07.287277 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2884b5-ce01-4ff3-a171-b10f6667234b None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:07.287691 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2884b5-ce01-4ff3-a171-b10f6667234b None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:07.287858 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2884b5-ce01-4ff3-a171-b10f6667234b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:07.304588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-1e2884b5-ce01-4ff3-a171-b10f6667234b None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:07.305818 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-1e2884b5-ce01-4ff3-a171-b10f6667234b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:07.390106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 505/1018] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:07.413307 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721184ad-09f7-472f-8a79-690799d99c0d None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:07.528549 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721184ad-09f7-472f-8a79-690799d99c0d None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:07.563860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e15b4-4d09-4f6f-9728-7f48a731d525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:07.564930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e15b4-4d09-4f6f-9728-7f48a731d525 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:07.565415 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e15b4-4d09-4f6f-9728-7f48a731d525 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:07.565891 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e15b4-4d09-4f6f-9728-7f48a731d525 None admin] PATH_INFO: `/v3/services/image` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:07.566723 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e15b4-4d09-4f6f-9728-7f48a731d525 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:07.568265 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e15b4-4d09-4f6f-9728-7f48a731d525 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:07.579902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-c63e15b4-4d09-4f6f-9728-7f48a731d525 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 20:27:07.579902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:07.579902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:27:07.579902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:27:07.579902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:27:07.579902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:27:07.579902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:27:07.579902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:27:07.579902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 20:27:07.579902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:07.579902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:27:07.579902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:07.579902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:07.579902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:07.579902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:07.579902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:07.579902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:07.580935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:07.580935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:07.580935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:07.580935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:07.580935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:07.580935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:07.580935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:27:07.580935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:27:07.580935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:27:07.580935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:27:07.580935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:07.580935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:07.580935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:07.580935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:07.580935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:07.580935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:07.581944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:07.581944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:07.581944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:07.581944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:07.581944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:07.581944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:07.581944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:07.581944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:07.581944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:07.581944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:07.581944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:27:07.581944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:27:07.581944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 20:27:07.581944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:07.587938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 514/1019] 10.222.0.71 () {56 vars in 1105 bytes} [Thu Jul 21 20:27:07 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:07.588691 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2884b5-ce01-4ff3-a171-b10f6667234b None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:07.692904 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2884b5-ce01-4ff3-a171-b10f6667234b None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:07.716932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:07.730898 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f7256e-8d83-44a5-b324-42bbecce3bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:07.731786 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7256e-8d83-44a5-b324-42bbecce3bc8 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:07.732222 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7256e-8d83-44a5-b324-42bbecce3bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:07.732680 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7256e-8d83-44a5-b324-42bbecce3bc8 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:07.733577 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f7256e-8d83-44a5-b324-42bbecce3bc8 None admin] RBAC: Adding query filter params (name=image) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:07.734053 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f7256e-8d83-44a5-b324-42bbecce3bc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:07.735582 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f7256e-8d83-44a5-b324-42bbecce3bc8 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:07.741095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 506/1020] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:27:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:07.747701 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e15b4-4d09-4f6f-9728-7f48a731d525 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:07.790787 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:07.851645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e15b4-4d09-4f6f-9728-7f48a731d525 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:07.886267 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9056842-36a7-471b-a55c-59778ca94e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:07.886928 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9056842-36a7-471b-a55c-59778ca94e91 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:07.887212 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9056842-36a7-471b-a55c-59778ca94e91 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:07.887486 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9056842-36a7-471b-a55c-59778ca94e91 None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:07.888192 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9056842-36a7-471b-a55c-59778ca94e91 None admin] RBAC: Adding query filter params (type=image) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:07.888502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9056842-36a7-471b-a55c-59778ca94e91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:07.889900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9056842-36a7-471b-a55c-59778ca94e91 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:07.899551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 515/1021] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:27:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:07.900749 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f7256e-8d83-44a5-b324-42bbecce3bc8 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:08.004672 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f7256e-8d83-44a5-b324-42bbecce3bc8 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:08.038917 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15193c45-3ee8-467f-9f04-1c70373967d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:08.039581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15193c45-3ee8-467f-9f04-1c70373967d1 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:08.039864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15193c45-3ee8-467f-9f04-1c70373967d1 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:08.040141 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15193c45-3ee8-467f-9f04-1c70373967d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:08.040947 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15193c45-3ee8-467f-9f04-1c70373967d1 None admin] RBAC: Adding query filter params (interface=public, service_id=58406d6fcc824119a72abd87e5d8fdf0, region_id=RegionOne) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:08.043184 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15193c45-3ee8-467f-9f04-1c70373967d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:08.043838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15193c45-3ee8-467f-9f04-1c70373967d1 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:08.056443 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 507/1022] 10.222.0.71 () {56 vars in 1251 bytes} [Thu Jul 21 20:27:07 2022] GET /identity/v3/endpoints?service_id=58406d6fcc824119a72abd87e5d8fdf0&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:08.058667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9056842-36a7-471b-a55c-59778ca94e91 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:08.167291 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9056842-36a7-471b-a55c-59778ca94e91 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:08.169677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:08.202768 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567b16a-b387-4ebe-8d19-50013104c5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:08.203504 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567b16a-b387-4ebe-8d19-50013104c5a4 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:08.203784 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567b16a-b387-4ebe-8d19-50013104c5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:08.204055 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567b16a-b387-4ebe-8d19-50013104c5a4 None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:08.204734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567b16a-b387-4ebe-8d19-50013104c5a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:08.206144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567b16a-b387-4ebe-8d19-50013104c5a4 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:08.212793 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 516/1023] 10.222.0.71 () {56 vars in 1087 bytes} [Thu Jul 21 20:27:08 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:27:08.284262 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab44ce0-1620-45b2-95ef-e00cbc3c3d47 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:08.284591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab44ce0-1620-45b2-95ef-e00cbc3c3d47 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:08.284850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab44ce0-1620-45b2-95ef-e00cbc3c3d47 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:08.286459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 508/1024] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:08.292738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be196a19-dd76-4102-9bdc-aa6a8dd96b63 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:08.293321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be196a19-dd76-4102-9bdc-aa6a8dd96b63 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:08.293321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be196a19-dd76-4102-9bdc-aa6a8dd96b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:08.311763 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-be196a19-dd76-4102-9bdc-aa6a8dd96b63 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:08.312997 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-be196a19-dd76-4102-9bdc-aa6a8dd96b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:08.386237 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 517/1025] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:08.409802 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab44ce0-1620-45b2-95ef-e00cbc3c3d47 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:08.465295 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:08.532829 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab44ce0-1620-45b2-95ef-e00cbc3c3d47 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:08.566931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf479c-2c5f-45fe-a0e2-c8655f8708a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:08.567721 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf479c-2c5f-45fe-a0e2-c8655f8708a3 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:08.568001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf479c-2c5f-45fe-a0e2-c8655f8708a3 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:08.568272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf479c-2c5f-45fe-a0e2-c8655f8708a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:08.569464 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacf479c-2c5f-45fe-a0e2-c8655f8708a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:08.570892 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacf479c-2c5f-45fe-a0e2-c8655f8708a3 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:08.572715 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 509/1026] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:27:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:08.578933 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be196a19-dd76-4102-9bdc-aa6a8dd96b63 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:08.675580 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:08.680917 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be196a19-dd76-4102-9bdc-aa6a8dd96b63 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:08.715995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb50438-7119-4d3d-b461-9bf18ad6f726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:08.716575 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb50438-7119-4d3d-b461-9bf18ad6f726 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:08.716848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb50438-7119-4d3d-b461-9bf18ad6f726 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:08.717141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb50438-7119-4d3d-b461-9bf18ad6f726 None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:08.717856 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb50438-7119-4d3d-b461-9bf18ad6f726 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:08.719273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb50438-7119-4d3d-b461-9bf18ad6f726 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:08.753115 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 518/1027] 10.222.0.71 () {60 vars in 1141 bytes} [Thu Jul 21 20:27:08 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:27:09.067456 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4329f-92b1-4080-b42b-1f36ce7876d4 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:09.067639 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4329f-92b1-4080-b42b-1f36ce7876d4 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:09.067925 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4329f-92b1-4080-b42b-1f36ce7876d4 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:09.069343 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 510/1028] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:09.074542 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf97392-4629-4093-a8dc-e982fe2ad041 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:09.074911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf97392-4629-4093-a8dc-e982fe2ad041 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:09.075265 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf97392-4629-4093-a8dc-e982fe2ad041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:09.092047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-faf97392-4629-4093-a8dc-e982fe2ad041 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:09.093321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-faf97392-4629-4093-a8dc-e982fe2ad041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:09.167131 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 519/1029] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:09.191586 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4329f-92b1-4080-b42b-1f36ce7876d4 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:09.302379 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4329f-92b1-4080-b42b-1f36ce7876d4 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:09.319882 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:09.341308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47248417-ee0e-444e-adad-3a1cd12d0403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:09.342151 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47248417-ee0e-444e-adad-3a1cd12d0403 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:09.342433 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47248417-ee0e-444e-adad-3a1cd12d0403 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:09.342705 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47248417-ee0e-444e-adad-3a1cd12d0403 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:09.354676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47248417-ee0e-444e-adad-3a1cd12d0403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:27:09.362863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-47248417-ee0e-444e-adad-3a1cd12d0403 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:27:09.362863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:09.362863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:09.362863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:09.362863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:09.362863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:09.362863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:09.362863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:09.362863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:09.362863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:09.362863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:09.362863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:09.362863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:27:09.362863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:27:09.362863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:27:09.362863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:27:09.363831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:27:09.363831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:27:09.363831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:27:09.363831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:27:09.363831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:09.363831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:09.363831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:09.363831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:09.363831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:09.363831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:09.363831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:09.363831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:09.363831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:09.363831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:09.363831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:09.363831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:09.364701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:09.364701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:09.364701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:09.364701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:09.364701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:27:09.364701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:27:09.364701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:27:09.364701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:27:09.364701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:27:09.364701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:27:09.364701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:27:09.364701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:09.364701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 511/1030] 10.222.0.71 () {56 vars in 1120 bytes} [Thu Jul 21 20:27:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:09.370059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf97392-4629-4093-a8dc-e982fe2ad041 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:09.456840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:09.485002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf97392-4629-4093-a8dc-e982fe2ad041 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:09.522446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e5ff6-a409-4436-a07e-4b1bb4fd79c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:09.523115 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e5ff6-a409-4436-a07e-4b1bb4fd79c9 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:09.523398 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e5ff6-a409-4436-a07e-4b1bb4fd79c9 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:09.523674 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e5ff6-a409-4436-a07e-4b1bb4fd79c9 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:09.524420 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e5ff6-a409-4436-a07e-4b1bb4fd79c9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:09.524779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e5ff6-a409-4436-a07e-4b1bb4fd79c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:09.526217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e5ff6-a409-4436-a07e-4b1bb4fd79c9 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:09.534685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 520/1031] 10.222.0.71 () {56 vars in 1115 bytes} [Thu Jul 21 20:27:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:09.541362 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47248417-ee0e-444e-adad-3a1cd12d0403 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:09.647649 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47248417-ee0e-444e-adad-3a1cd12d0403 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:09.681780 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355ddd6-89d2-4b1f-b361-96ed996e7252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:09.683103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355ddd6-89d2-4b1f-b361-96ed996e7252 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:09.683380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355ddd6-89d2-4b1f-b361-96ed996e7252 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:09.683653 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355ddd6-89d2-4b1f-b361-96ed996e7252 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:09.694942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355ddd6-89d2-4b1f-b361-96ed996e7252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:09.696383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355ddd6-89d2-4b1f-b361-96ed996e7252 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:09.714070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-b355ddd6-89d2-4b1f-b361-96ed996e7252 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:27:09.714070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:09.714070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:09.714070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:09.714070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:09.714070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:09.714070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:09.714070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:09.714070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:09.714070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:09.714070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:09.714070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:09.714070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:27:09.714070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:27:09.714070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:27:09.715004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:27:09.715004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:09.715004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:09.715004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:09.715004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:09.715004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:09.715004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:09.715004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:09.715004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:09.715004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:09.715004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:09.715004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:09.715004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:09.715004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:09.715004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:09.715004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:09.715004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:09.715970 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:27:09.715970 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:27:09.715970 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:27:09.715970 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:09.715970 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 512/1032] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:27:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:09.721973 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e5ff6-a409-4436-a07e-4b1bb4fd79c9 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:09.828498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e5ff6-a409-4436-a07e-4b1bb4fd79c9 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:09.863855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691553f-7fc6-4fa6-9c5d-9d5628b6d9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:09.864518 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691553f-7fc6-4fa6-9c5d-9d5628b6d9a1 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:09.864802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691553f-7fc6-4fa6-9c5d-9d5628b6d9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:09.865097 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691553f-7fc6-4fa6-9c5d-9d5628b6d9a1 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:09.865828 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691553f-7fc6-4fa6-9c5d-9d5628b6d9a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:09.866138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691553f-7fc6-4fa6-9c5d-9d5628b6d9a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:09.867514 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691553f-7fc6-4fa6-9c5d-9d5628b6d9a1 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:09.884724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 521/1033] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:09.886103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355ddd6-89d2-4b1f-b361-96ed996e7252 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:09.920283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:09.995351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355ddd6-89d2-4b1f-b361-96ed996e7252 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:10.029971 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c0c0f-4d4a-4100-bc19-8b40c07892d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:10.030753 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c0c0f-4d4a-4100-bc19-8b40c07892d5 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:10.031032 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c0c0f-4d4a-4100-bc19-8b40c07892d5 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:10.031304 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c0c0f-4d4a-4100-bc19-8b40c07892d5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:10.039267 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c0c0f-4d4a-4100-bc19-8b40c07892d5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:10.040711 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c0c0f-4d4a-4100-bc19-8b40c07892d5 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:10.054837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-335c0c0f-4d4a-4100-bc19-8b40c07892d5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:27:10.054837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:10.054837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:10.054837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:10.054837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:10.054837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:10.054837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:10.054837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:10.054837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:10.054837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:10.054837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:10.054837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:10.054837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:27:10.054837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:27:10.054837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:27:10.054837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:27:10.055823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:10.055823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:10.055823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:27:10.055823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:27:10.055823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:27:10.055823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:27:10.055823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:10.055823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:10.055823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:10.055823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:10.055823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:10.055823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:10.055823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:10.055823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:10.055823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:10.055823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:10.055823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:10.055823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:10.056773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:10.056773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:10.056773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:27:10.056773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:27:10.056773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:27:10.056773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:27:10.056773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:27:10.056773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:27:10.056773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:27:10.056773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:10.061270 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 513/1034] 10.222.0.71 () {56 vars in 1093 bytes} [Thu Jul 21 20:27:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:10.063141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691553f-7fc6-4fa6-9c5d-9d5628b6d9a1 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:10.067747 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:10.174528 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691553f-7fc6-4fa6-9c5d-9d5628b6d9a1 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:10.207381 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad3635-8307-43e7-8cdd-bbc51c92146d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:10.208042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad3635-8307-43e7-8cdd-bbc51c92146d None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:10.208322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad3635-8307-43e7-8cdd-bbc51c92146d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:10.208594 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad3635-8307-43e7-8cdd-bbc51c92146d None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:10.209454 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad3635-8307-43e7-8cdd-bbc51c92146d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:10.209762 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad3635-8307-43e7-8cdd-bbc51c92146d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:10.211169 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad3635-8307-43e7-8cdd-bbc51c92146d None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:10.234140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 522/1035] 10.222.0.71 () {56 vars in 1133 bytes} [Thu Jul 21 20:27:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:10.235971 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c0c0f-4d4a-4100-bc19-8b40c07892d5 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:10.278032 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa39f5-8e52-4a73-a6f4-c081d73d8e59 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:10.278370 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa39f5-8e52-4a73-a6f4-c081d73d8e59 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:10.278574 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa39f5-8e52-4a73-a6f4-c081d73d8e59 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:10.279937 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 523/1036] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:27:10.285233 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883eccb-4d31-432f-92d4-6056eef08466 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:10.285518 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883eccb-4d31-432f-92d4-6056eef08466 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:10.285833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883eccb-4d31-432f-92d4-6056eef08466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:10.303266 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-3883eccb-4d31-432f-92d4-6056eef08466 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:10.304439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3883eccb-4d31-432f-92d4-6056eef08466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:10.335140 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:10.346876 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c0c0f-4d4a-4100-bc19-8b40c07892d5 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:10.366489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:10.380151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 524/1037] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:27:10.385098 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cd968-6f3a-4146-a4e1-a1ef64041470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:10.385820 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cd968-6f3a-4146-a4e1-a1ef64041470 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:10.386105 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cd968-6f3a-4146-a4e1-a1ef64041470 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:10.386405 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cd968-6f3a-4146-a4e1-a1ef64041470 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:10.403533 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883eccb-4d31-432f-92d4-6056eef08466 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:10.463019 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936cd968-6f3a-4146-a4e1-a1ef64041470 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:10.465015 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936cd968-6f3a-4146-a4e1-a1ef64041470 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:10.517347 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883eccb-4d31-432f-92d4-6056eef08466 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:10.554589 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504422a-8949-4a5b-8180-2ee8a7d1907c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:10.555381 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504422a-8949-4a5b-8180-2ee8a7d1907c None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:10.555663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504422a-8949-4a5b-8180-2ee8a7d1907c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:10.555935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504422a-8949-4a5b-8180-2ee8a7d1907c None admin] PATH_INFO: `/v3/services/image` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:10.556591 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504422a-8949-4a5b-8180-2ee8a7d1907c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:10.558010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504422a-8949-4a5b-8180-2ee8a7d1907c None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:10.566467 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:10.570391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-0504422a-8949-4a5b-8180-2ee8a7d1907c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 20:27:10.570391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:10.570391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:27:10.570391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:27:10.570391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:27:10.570391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:27:10.570391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:27:10.570391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:27:10.570391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 20:27:10.570391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:10.570391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:27:10.570391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:10.570391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:10.570391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:10.570391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:10.570391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:10.570391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:10.571426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:10.571426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:10.571426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:10.571426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:10.571426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:10.571426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:10.571426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:27:10.571426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:27:10.571426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:27:10.571426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:27:10.571426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:10.571426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:10.571426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:10.571426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:10.571426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:10.571426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:10.572308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:10.572308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:10.572308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:10.572308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:10.572308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:10.572308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:10.572308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:10.572308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:10.572308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:10.572308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:10.572308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:27:10.572308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:27:10.572308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 20:27:10.572308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:10.572308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 525/1038] 10.222.0.71 () {56 vars in 1105 bytes} [Thu Jul 21 20:27:10 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:27:10.577902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504422a-8949-4a5b-8180-2ee8a7d1907c None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:10.582174 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 514/1039] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:27:10 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 20:27:10.587369 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cd968-6f3a-4146-a4e1-a1ef64041470 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:10.643877 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:10.692629 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cd968-6f3a-4146-a4e1-a1ef64041470 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:10.693817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504422a-8949-4a5b-8180-2ee8a7d1907c None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:10.729617 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ae0f4-a277-4b8e-896c-c0924be7c953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:10.730324 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ae0f4-a277-4b8e-896c-c0924be7c953 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:10.730608 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ae0f4-a277-4b8e-896c-c0924be7c953 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:10.730709 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a3052-827d-4a5d-ae2a-8455b95501bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:10.730898 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ae0f4-a277-4b8e-896c-c0924be7c953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:10.731279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a3052-827d-4a5d-ae2a-8455b95501bf None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:10.731564 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a3052-827d-4a5d-ae2a-8455b95501bf None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:10.731839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a3052-827d-4a5d-ae2a-8455b95501bf None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:10.732555 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a3052-827d-4a5d-ae2a-8455b95501bf None admin] RBAC: Adding query filter params (name=image) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:10.732872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a3052-827d-4a5d-ae2a-8455b95501bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:10.734280 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a3052-827d-4a5d-ae2a-8455b95501bf None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:10.739565 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 526/1040] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:27:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:10.742189 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ae0f4-a277-4b8e-896c-c0924be7c953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:10.743650 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ae0f4-a277-4b8e-896c-c0924be7c953 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:10.745308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a3052-827d-4a5d-ae2a-8455b95501bf None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:10.763377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-ee6ae0f4-a277-4b8e-896c-c0924be7c953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:27:10.763377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:10.763377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:10.763377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:10.763377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:10.763377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:10.763377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:10.763377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:10.763377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:10.763377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:10.763377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:10.763377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:10.763377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:27:10.763377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:27:10.763377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:27:10.764349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:27:10.764349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:10.764349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:10.764349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:10.764349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:10.764349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:10.764349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:10.764349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:10.764349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:10.764349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:10.764349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:10.764349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:10.764349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:10.764349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:10.764349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:10.764349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:10.764349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:10.765501 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:27:10.765501 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:27:10.765501 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:27:10.765501 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:10.765501 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 515/1041] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:27:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:27:10.770199 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ae0f4-a277-4b8e-896c-c0924be7c953 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:10.817133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:10.853696 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a3052-827d-4a5d-ae2a-8455b95501bf None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:10.871527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:10.879028 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ae0f4-a277-4b8e-896c-c0924be7c953 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:10.888628 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5620e5-ef4f-464d-a211-a9c7af49a324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:10.889316 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5620e5-ef4f-464d-a211-a9c7af49a324 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:10.889606 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5620e5-ef4f-464d-a211-a9c7af49a324 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:10.889891 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5620e5-ef4f-464d-a211-a9c7af49a324 None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:10.890588 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5620e5-ef4f-464d-a211-a9c7af49a324 None admin] RBAC: Adding query filter params (type=image) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:10.890900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5620e5-ef4f-464d-a211-a9c7af49a324 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:10.892271 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5620e5-ef4f-464d-a211-a9c7af49a324 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:10.897161 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 527/1042] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:27:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:10.898373 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f14ce-f86b-490a-af11-e053d35d536f None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:10.898660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f14ce-f86b-490a-af11-e053d35d536f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:10.898938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f14ce-f86b-490a-af11-e053d35d536f None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:10.900164 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 528/1043] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:10.905450 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f14ce-f86b-490a-af11-e053d35d536f None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:10.919781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ae1cf-377b-472e-aa2f-97aefcb3e003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:10.920447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ae1cf-377b-472e-aa2f-97aefcb3e003 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:10.920731 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ae1cf-377b-472e-aa2f-97aefcb3e003 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:10.921010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ae1cf-377b-472e-aa2f-97aefcb3e003 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:10.921768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ae1cf-377b-472e-aa2f-97aefcb3e003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:10.922081 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ae1cf-377b-472e-aa2f-97aefcb3e003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:10.923465 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ae1cf-377b-472e-aa2f-97aefcb3e003 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:10.934754 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 516/1044] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:10.938499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ac627-4a12-4fd2-beb0-553e15d10149 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:10.938729 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ac627-4a12-4fd2-beb0-553e15d10149 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:10.939007 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ac627-4a12-4fd2-beb0-553e15d10149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:10.956507 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-ef2ac627-4a12-4fd2-beb0-553e15d10149 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:10.957854 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-ef2ac627-4a12-4fd2-beb0-553e15d10149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:11.012727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f14ce-f86b-490a-af11-e053d35d536f None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:11.025058 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 517/1045] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:11.026456 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ac627-4a12-4fd2-beb0-553e15d10149 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:11.050645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90379729-d9eb-49f2-a73c-7599ad9a915a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:11.051300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90379729-d9eb-49f2-a73c-7599ad9a915a None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:11.051582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90379729-d9eb-49f2-a73c-7599ad9a915a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:11.051857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90379729-d9eb-49f2-a73c-7599ad9a915a None admin] PATH_INFO: `/v3/endpoints` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:11.052605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90379729-d9eb-49f2-a73c-7599ad9a915a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:11.054024 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90379729-d9eb-49f2-a73c-7599ad9a915a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:11.066192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:11.072516 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 529/1046] 10.222.0.71 () {60 vars in 1144 bytes} [Thu Jul 21 20:27:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:27:11.073985 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90379729-d9eb-49f2-a73c-7599ad9a915a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:11.121106 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:11.152675 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ac627-4a12-4fd2-beb0-553e15d10149 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:11.196122 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3756686-759a-4288-8c00-52a2a61c838a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:11.196927 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3756686-759a-4288-8c00-52a2a61c838a None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:11.197238 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3756686-759a-4288-8c00-52a2a61c838a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:11.197507 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3756686-759a-4288-8c00-52a2a61c838a None admin] PATH_INFO: `/v3/projects/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:11.202471 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90379729-d9eb-49f2-a73c-7599ad9a915a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:11.204859 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3756686-759a-4288-8c00-52a2a61c838a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:11.206358 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3756686-759a-4288-8c00-52a2a61c838a None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:11.221311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-b3756686-759a-4288-8c00-52a2a61c838a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:27:11.221311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:11.221311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:11.221311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:11.221311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:11.221311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:11.221311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:11.221311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:11.221311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:11.221311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:11.221311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:11.221311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:11.221311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:27:11.221311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:27:11.221311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:27:11.222289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:27:11.222289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:11.222289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:11.222289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:11.222289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:11.222289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:11.222289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:11.222289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:11.222289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:11.222289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:11.222289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:11.222289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:11.222289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:11.222289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:11.222289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:11.222289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:11.222289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:11.223499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:27:11.223499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:27:11.223499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:27:11.223499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:27:11.223499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:27:11.223499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:27:11.223499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:27:11.223499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:11.223499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 518/1047] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:27:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:11.228960 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3756686-759a-4288-8c00-52a2a61c838a None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:11.244360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b679407-65b0-4265-8327-61e3e402a97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:11.245447 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b679407-65b0-4265-8327-61e3e402a97e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:11.246427 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b679407-65b0-4265-8327-61e3e402a97e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:11.246427 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b679407-65b0-4265-8327-61e3e402a97e None admin] PATH_INFO: `/v3/domains/default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:11.247168 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b679407-65b0-4265-8327-61e3e402a97e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:11.248624 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b679407-65b0-4265-8327-61e3e402a97e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:11.250442 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 530/1048] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:27:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:11.256707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b679407-65b0-4265-8327-61e3e402a97e None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:11.329216 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:11.343663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3756686-759a-4288-8c00-52a2a61c838a None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:11.369351 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b679407-65b0-4265-8327-61e3e402a97e None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:11.371452 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:11.380058 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9c074-bf56-4422-99a8-010477b6eb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:11.380718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9c074-bf56-4422-99a8-010477b6eb5d None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:11.380999 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9c074-bf56-4422-99a8-010477b6eb5d None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:11.381312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9c074-bf56-4422-99a8-010477b6eb5d None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:11.382108 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9c074-bf56-4422-99a8-010477b6eb5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:11.382424 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9c074-bf56-4422-99a8-010477b6eb5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:11.383862 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9c074-bf56-4422-99a8-010477b6eb5d None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:11.402588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 519/1049] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:27:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:11.403492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9c074-bf56-4422-99a8-010477b6eb5d None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:11.408964 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770a538-a5b5-4359-a28c-80493c3c6045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:11.410250 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770a538-a5b5-4359-a28c-80493c3c6045 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:11.410738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770a538-a5b5-4359-a28c-80493c3c6045 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:11.411196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770a538-a5b5-4359-a28c-80493c3c6045 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:11.412104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770a538-a5b5-4359-a28c-80493c3c6045 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:11.413590 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770a538-a5b5-4359-a28c-80493c3c6045 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:11.445110 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 531/1050] 10.222.0.71 () {60 vars in 1132 bytes} [Thu Jul 21 20:27:11 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:27:11.536570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9c074-bf56-4422-99a8-010477b6eb5d None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:11.574216 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16eebe-9a3c-441b-8650-7ecce7e23331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:11.574907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16eebe-9a3c-441b-8650-7ecce7e23331 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:11.575153 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16eebe-9a3c-441b-8650-7ecce7e23331 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:11.575431 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16eebe-9a3c-441b-8650-7ecce7e23331 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:11.576288 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16eebe-9a3c-441b-8650-7ecce7e23331 None admin] RBAC: Adding query filter params (role.id=4eefa64717a84653822b16532ebf6e75, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=6fbc5240f5214aaea4418052b94739e2) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:11.576603 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16eebe-9a3c-441b-8650-7ecce7e23331 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:11.578128 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16eebe-9a3c-441b-8650-7ecce7e23331 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:11.593249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 520/1051] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:27:11 2022] GET /identity/v3/role_assignments?user.id=6fbc5240f5214aaea4418052b94739e2&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=4eefa64717a84653822b16532ebf6e75 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:13.150535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9772a1-a6e1-4548-9001-a516115be3b4 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:13.151606 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9772a1-a6e1-4548-9001-a516115be3b4 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:13.151720 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9772a1-a6e1-4548-9001-a516115be3b4 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:13.156859 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 532/1052] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:13.159425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26345b1b-dc9c-448c-88af-85b81ca5ec2e None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:13.159873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26345b1b-dc9c-448c-88af-85b81ca5ec2e None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:13.160792 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26345b1b-dc9c-448c-88af-85b81ca5ec2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:13.178993 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-26345b1b-dc9c-448c-88af-85b81ca5ec2e None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:13.180172 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-26345b1b-dc9c-448c-88af-85b81ca5ec2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:13.188485 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:13.253123 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 521/1053] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:13.273704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9772a1-a6e1-4548-9001-a516115be3b4 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:13.379438 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9772a1-a6e1-4548-9001-a516115be3b4 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:13.413499 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b7a94-d380-4510-b2c3-ab4ce83320cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:13.414154 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b7a94-d380-4510-b2c3-ab4ce83320cd None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:13.414438 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b7a94-d380-4510-b2c3-ab4ce83320cd None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:13.414718 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b7a94-d380-4510-b2c3-ab4ce83320cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:13.442450 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:13.493060 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b7a94-d380-4510-b2c3-ab4ce83320cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:13.495032 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b7a94-d380-4510-b2c3-ab4ce83320cd None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:13.558067 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b6ac5-1992-4c02-893f-c995d86d8b5a None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:13.558265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b6ac5-1992-4c02-893f-c995d86d8b5a None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:13.558538 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b6ac5-1992-4c02-893f-c995d86d8b5a None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:13.560228 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 522/1054] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:27:13.566523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8a960-2df0-4708-bc12-e389b3908964 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:13.566523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8a960-2df0-4708-bc12-e389b3908964 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:13.566523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8a960-2df0-4708-bc12-e389b3908964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:13.583470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-b9a8a960-2df0-4708-bc12-e389b3908964 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:13.584667 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-b9a8a960-2df0-4708-bc12-e389b3908964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:13.605755 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 533/1055] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:27:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:27:13.611430 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b7a94-d380-4510-b2c3-ab4ce83320cd None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:13.662943 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 523/1056] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:27:13.665351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384fb90-9c3d-4c49-82eb-55b62392b3aa None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:13.665351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384fb90-9c3d-4c49-82eb-55b62392b3aa None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:13.665351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384fb90-9c3d-4c49-82eb-55b62392b3aa None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:13.666460 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 524/1057] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:13.674658 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a224949-a027-497a-a82c-9e6733228456 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:13.675109 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a224949-a027-497a-a82c-9e6733228456 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:13.675297 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a224949-a027-497a-a82c-9e6733228456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:13.676328 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:13.692716 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-0a224949-a027-497a-a82c-9e6733228456 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:13.693948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-0a224949-a027-497a-a82c-9e6733228456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:13.730463 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b7a94-d380-4510-b2c3-ab4ce83320cd None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:13.769884 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602ea03-a7af-4e4c-ae4e-1788a34ce353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:13.770668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602ea03-a7af-4e4c-ae4e-1788a34ce353 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:13.770766 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 525/1058] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:27:13.770943 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602ea03-a7af-4e4c-ae4e-1788a34ce353 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:13.771217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602ea03-a7af-4e4c-ae4e-1788a34ce353 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:13.771768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a224949-a027-497a-a82c-9e6733228456 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:13.774020 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:13.787169 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602ea03-a7af-4e4c-ae4e-1788a34ce353 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:13.788643 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602ea03-a7af-4e4c-ae4e-1788a34ce353 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:13.807329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-a602ea03-a7af-4e4c-ae4e-1788a34ce353 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:27:13.807329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:13.807329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:13.807329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:13.807329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:13.807329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:13.807329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:13.807329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:13.807329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:13.807329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:13.807329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:13.807329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:13.807329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:27:13.807329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:27:13.807329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:27:13.808409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:27:13.808409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:13.808409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:13.808409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:13.808409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:13.808409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:13.808409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:13.808409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:13.808409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:13.808409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:13.808409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:13.808409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:13.808409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:13.808409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:13.808409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:13.808409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:13.808409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:13.809986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:27:13.809986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:27:13.809986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:27:13.809986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:13.809986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 534/1059] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:27:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:27:13.809986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602ea03-a7af-4e4c-ae4e-1788a34ce353 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:13.880742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a224949-a027-497a-a82c-9e6733228456 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:13.915548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602ea03-a7af-4e4c-ae4e-1788a34ce353 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:13.918552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38ccdd-2657-4a62-b215-db150dbe706b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:13.919347 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38ccdd-2657-4a62-b215-db150dbe706b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:13.919621 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38ccdd-2657-4a62-b215-db150dbe706b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:13.919891 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38ccdd-2657-4a62-b215-db150dbe706b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:13.926244 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:13.933876 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38ccdd-2657-4a62-b215-db150dbe706b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:27:13.941526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-6f38ccdd-2657-4a62-b215-db150dbe706b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:27:13.941526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:13.941526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:13.941526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:13.941526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:13.941526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:13.941526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:13.941526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:13.941526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:13.941526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:13.941526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:13.941526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:13.941526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:27:13.941526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:27:13.941526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:27:13.941526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:27:13.943119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:27:13.943119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:27:13.943119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:27:13.943119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:27:13.943119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:13.943119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:13.943119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:13.943119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:13.943119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:13.943119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:13.943119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:13.943119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:13.943119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:13.943119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:13.943119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:13.943119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:13.943984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:13.943984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:13.943984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:13.943984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:13.943984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:27:13.943984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:27:13.943984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:27:13.943984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:27:13.943984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:27:13.943984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:27:13.943984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:27:13.943984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:13.943984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 526/1060] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:27:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:13.943984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38ccdd-2657-4a62-b215-db150dbe706b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:13.962658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6871b6a-009b-4c26-ad05-6375329e7660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:13.963535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6871b6a-009b-4c26-ad05-6375329e7660 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:13.963820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6871b6a-009b-4c26-ad05-6375329e7660 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:13.964089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6871b6a-009b-4c26-ad05-6375329e7660 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:13.975743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6871b6a-009b-4c26-ad05-6375329e7660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:27:13.984075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-e6871b6a-009b-4c26-ad05-6375329e7660 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:27:13.984075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:13.984075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:13.984075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:13.984075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:13.984075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:13.984075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:13.984075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:13.984075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:13.984075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:13.984075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:13.984075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:13.984075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:27:13.984075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:27:13.984075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:27:13.984075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:27:13.985294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:27:13.985294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:27:13.985294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:27:13.985294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:27:13.985294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:13.985294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:13.985294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:13.985294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:13.985294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:13.985294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:13.985294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:13.985294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:13.985294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:13.985294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:13.985294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:13.985294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:13.986193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:13.986193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:13.986193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:13.986193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:13.986193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:27:13.986193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:27:13.986193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:27:13.986193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:27:13.986193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:27:13.986193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:27:13.986193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:27:13.986193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:13.986193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 535/1061] 10.222.0.71 () {56 vars in 1120 bytes} [Thu Jul 21 20:27:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:13.988292 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6871b6a-009b-4c26-ad05-6375329e7660 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:14.024109 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:14.064822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38ccdd-2657-4a62-b215-db150dbe706b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:14.096278 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6871b6a-009b-4c26-ad05-6375329e7660 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:14.102834 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a230d-b414-4e3c-bd8c-2bde02bad022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:14.103633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a230d-b414-4e3c-bd8c-2bde02bad022 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:14.104109 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a230d-b414-4e3c-bd8c-2bde02bad022 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:14.104589 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a230d-b414-4e3c-bd8c-2bde02bad022 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:14.105451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a230d-b414-4e3c-bd8c-2bde02bad022 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:14.105827 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a230d-b414-4e3c-bd8c-2bde02bad022 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:14.107524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a230d-b414-4e3c-bd8c-2bde02bad022 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:14.122491 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 527/1062] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:14.123350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a230d-b414-4e3c-bd8c-2bde02bad022 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:14.151991 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a974f19-9e61-4120-b76d-2722bff99212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:14.152732 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a974f19-9e61-4120-b76d-2722bff99212 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:14.153034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a974f19-9e61-4120-b76d-2722bff99212 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:14.153376 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a974f19-9e61-4120-b76d-2722bff99212 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:14.154280 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a974f19-9e61-4120-b76d-2722bff99212 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:14.154599 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a974f19-9e61-4120-b76d-2722bff99212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:14.156063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a974f19-9e61-4120-b76d-2722bff99212 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:14.169400 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 536/1063] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:27:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:14.179698 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a974f19-9e61-4120-b76d-2722bff99212 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:14.228690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:14.284263 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a230d-b414-4e3c-bd8c-2bde02bad022 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:14.314620 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:14.336938 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f4b14-b6dc-4fbe-8e4f-0e36ce3270fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:14.337813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f4b14-b6dc-4fbe-8e4f-0e36ce3270fa None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:14.338603 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f4b14-b6dc-4fbe-8e4f-0e36ce3270fa None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:14.338961 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f4b14-b6dc-4fbe-8e4f-0e36ce3270fa None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:14.340041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f4b14-b6dc-4fbe-8e4f-0e36ce3270fa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:14.340627 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f4b14-b6dc-4fbe-8e4f-0e36ce3270fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:14.342100 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f4b14-b6dc-4fbe-8e4f-0e36ce3270fa None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:14.346290 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a974f19-9e61-4120-b76d-2722bff99212 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:14.351372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 528/1064] 10.222.0.71 () {56 vars in 1115 bytes} [Thu Jul 21 20:27:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:14.358624 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f4b14-b6dc-4fbe-8e4f-0e36ce3270fa None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:14.388359 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f7459-98ff-4ecc-8249-3d5e8cd05ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:14.389286 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f7459-98ff-4ecc-8249-3d5e8cd05ef7 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:14.389569 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f7459-98ff-4ecc-8249-3d5e8cd05ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:14.389843 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f7459-98ff-4ecc-8249-3d5e8cd05ef7 None admin] PATH_INFO: `/v3/users/8e9363c137754c108d622f5efca3af81` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:14.409744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f7459-98ff-4ecc-8249-3d5e8cd05ef7 None admin] RBAC: Authorizing `identity:get_user(user_id=8e9363c137754c108d622f5efca3af81)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:14.409744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f7459-98ff-4ecc-8249-3d5e8cd05ef7 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:14.411791 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 537/1065] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:27:14 2022] GET /identity/v3/users/8e9363c137754c108d622f5efca3af81 => generated 285 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:14.418081 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f7459-98ff-4ecc-8249-3d5e8cd05ef7 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:14.512864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f4b14-b6dc-4fbe-8e4f-0e36ce3270fa None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:14.532335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:14.543545 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f7459-98ff-4ecc-8249-3d5e8cd05ef7 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:14.547190 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d92bf-974d-4614-a1f1-48fc585bb8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:14.547978 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d92bf-974d-4614-a1f1-48fc585bb8ce None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:14.548260 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d92bf-974d-4614-a1f1-48fc585bb8ce None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:14.548532 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d92bf-974d-4614-a1f1-48fc585bb8ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:14.559545 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d92bf-974d-4614-a1f1-48fc585bb8ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:14.560983 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d92bf-974d-4614-a1f1-48fc585bb8ce None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:14.577484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a74d75-dddd-4473-aa5d-6a93b1596878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:14.578122 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a74d75-dddd-4473-aa5d-6a93b1596878 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:14.578227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-148d92bf-974d-4614-a1f1-48fc585bb8ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:27:14.578227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:14.578227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:14.578227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:14.578227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:14.578227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:14.578227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:14.578227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:14.578227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:14.578227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:14.578227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:14.578227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:14.578227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:27:14.578227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:27:14.578227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:27:14.579192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:27:14.579192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:14.579192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:14.579192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:14.579192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:14.579192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:14.579192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:14.579192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:14.579192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:14.579192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:14.579192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:14.579192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:14.579192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:14.579192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:14.579192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:14.579192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:14.579192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:14.580437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:27:14.580437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:27:14.580437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:27:14.580437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:14.580730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a74d75-dddd-4473-aa5d-6a93b1596878 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:14.580730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a74d75-dddd-4473-aa5d-6a93b1596878 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:14.580893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 529/1066] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:27:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:27:14.585206 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d92bf-974d-4614-a1f1-48fc585bb8ce None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:14.620420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:14.648602 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a74d75-dddd-4473-aa5d-6a93b1596878 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:14.650079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a74d75-dddd-4473-aa5d-6a93b1596878 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:14.686500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d92bf-974d-4614-a1f1-48fc585bb8ce None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:14.720623 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8660b38-1e73-45d1-aa23-3fc60a1d3f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:14.721259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8660b38-1e73-45d1-aa23-3fc60a1d3f15 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:14.721515 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8660b38-1e73-45d1-aa23-3fc60a1d3f15 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:14.721786 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8660b38-1e73-45d1-aa23-3fc60a1d3f15 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:14.722503 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8660b38-1e73-45d1-aa23-3fc60a1d3f15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:14.722828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8660b38-1e73-45d1-aa23-3fc60a1d3f15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:14.724417 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8660b38-1e73-45d1-aa23-3fc60a1d3f15 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:14.736606 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 530/1067] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:14.741817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:14.742007 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8660b38-1e73-45d1-aa23-3fc60a1d3f15 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:14.750437 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 538/1068] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:27:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:27:14.756100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a74d75-dddd-4473-aa5d-6a93b1596878 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:14.838644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:14.845053 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8660b38-1e73-45d1-aa23-3fc60a1d3f15 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:14.855268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a74d75-dddd-4473-aa5d-6a93b1596878 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:14.881049 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44639a72-5d35-4ab8-a393-cc99092a4df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:14.881913 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44639a72-5d35-4ab8-a393-cc99092a4df7 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:14.882195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44639a72-5d35-4ab8-a393-cc99092a4df7 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:14.882593 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44639a72-5d35-4ab8-a393-cc99092a4df7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:14.889936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44639a72-5d35-4ab8-a393-cc99092a4df7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:14.890907 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2ed50-7cb9-4266-8bb2-9d90d2c61c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:14.891371 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44639a72-5d35-4ab8-a393-cc99092a4df7 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:14.891762 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2ed50-7cb9-4266-8bb2-9d90d2c61c54 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:14.892042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2ed50-7cb9-4266-8bb2-9d90d2c61c54 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:14.892313 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2ed50-7cb9-4266-8bb2-9d90d2c61c54 None admin] PATH_INFO: `/v3/projects/886e3d01e51c4c759a9de90e182f61db` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:14.893586 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2ed50-7cb9-4266-8bb2-9d90d2c61c54 None admin] RBAC: Authorizing `identity:get_project(project_id=886e3d01e51c4c759a9de90e182f61db)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:14.895013 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2ed50-7cb9-4266-8bb2-9d90d2c61c54 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:14.896727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 539/1069] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:27:14 2022] GET /identity/v3/projects/886e3d01e51c4c759a9de90e182f61db => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:14.903314 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2ed50-7cb9-4266-8bb2-9d90d2c61c54 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:14.906305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-44639a72-5d35-4ab8-a393-cc99092a4df7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:27:14.906305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:14.906305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:14.906305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:14.906305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:14.906305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:14.906305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:14.906305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:14.906305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:14.906305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:14.906305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:14.906305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:14.906305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:27:14.906305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:27:14.906305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:27:14.906305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:27:14.907422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:14.907422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:14.907422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:27:14.907422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:27:14.907422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:27:14.907422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:27:14.907422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:14.907422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:14.907422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:14.907422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:14.907422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:14.907422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:14.907422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:14.907422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:14.907422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:14.907422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:14.907422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:14.907422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:14.908633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:14.908633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:14.908633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:27:14.908633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:27:14.908633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:27:14.908633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:27:14.908633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:27:14.908633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:27:14.908633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:27:14.908633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:14.908633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 531/1070] 10.222.0.71 () {56 vars in 1093 bytes} [Thu Jul 21 20:27:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:14.913141 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44639a72-5d35-4ab8-a393-cc99092a4df7 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:14.965163 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:15.008357 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2ed50-7cb9-4266-8bb2-9d90d2c61c54 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:15.017866 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44639a72-5d35-4ab8-a393-cc99092a4df7 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:15.042267 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca564684-feb9-4734-998a-db97dbb7c129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:15.042912 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca564684-feb9-4734-998a-db97dbb7c129 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:15.043190 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca564684-feb9-4734-998a-db97dbb7c129 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:15.043464 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca564684-feb9-4734-998a-db97dbb7c129 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:15.044300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca564684-feb9-4734-998a-db97dbb7c129 None admin] RBAC: Adding query filter params (role.id=19f5e72c0be44a1197c7288de82e79e8, scope.project.id=886e3d01e51c4c759a9de90e182f61db, user.id=8e9363c137754c108d622f5efca3af81) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:15.044586 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca564684-feb9-4734-998a-db97dbb7c129 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:15.046037 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca564684-feb9-4734-998a-db97dbb7c129 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:15.051332 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 540/1071] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:27:14 2022] GET /identity/v3/role_assignments?user.id=8e9363c137754c108d622f5efca3af81&scope.project.id=886e3d01e51c4c759a9de90e182f61db&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:15.052787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a5e51-a3d9-4d84-931c-03acc3084dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:15.053487 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a5e51-a3d9-4d84-931c-03acc3084dff None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:15.053774 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a5e51-a3d9-4d84-931c-03acc3084dff None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:15.054054 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a5e51-a3d9-4d84-931c-03acc3084dff None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:15.054914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a5e51-a3d9-4d84-931c-03acc3084dff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:15.055231 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a5e51-a3d9-4d84-931c-03acc3084dff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:15.056700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a5e51-a3d9-4d84-931c-03acc3084dff None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:15.078298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 532/1072] 10.222.0.71 () {56 vars in 1133 bytes} [Thu Jul 21 20:27:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:15.084382 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca564684-feb9-4734-998a-db97dbb7c129 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:15.191436 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca564684-feb9-4734-998a-db97dbb7c129 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:15.215117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:15.227599 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc24eb-1b82-4e13-854c-e00212313e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:15.228384 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc24eb-1b82-4e13-854c-e00212313e5e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:15.228664 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc24eb-1b82-4e13-854c-e00212313e5e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:15.228939 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc24eb-1b82-4e13-854c-e00212313e5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:15.239809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc24eb-1b82-4e13-854c-e00212313e5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:15.241397 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc24eb-1b82-4e13-854c-e00212313e5e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:15.258348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-91fc24eb-1b82-4e13-854c-e00212313e5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:27:15.258348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:15.258348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:15.258348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:15.258348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:15.258348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:15.258348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:15.258348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:15.258348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:15.258348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:15.258348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:15.258348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:15.258348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:27:15.258348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:27:15.258348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:27:15.259484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:27:15.259484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:15.259484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:15.259484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:15.259484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:15.259484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:15.259484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:15.259484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:15.259484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:15.259484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:15.259484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:15.259484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:15.259484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:15.259484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:15.259484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:15.259484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:15.259484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:15.260950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:27:15.260950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:27:15.260950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:27:15.260950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:15.260950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 541/1073] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:27:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:15.265771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a5e51-a3d9-4d84-931c-03acc3084dff None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:15.282349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:15.367656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a5e51-a3d9-4d84-931c-03acc3084dff None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:15.402041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb4805-62f2-496e-8997-a7f877a77b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:15.402708 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb4805-62f2-496e-8997-a7f877a77b58 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:15.402989 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb4805-62f2-496e-8997-a7f877a77b58 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:15.403265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb4805-62f2-496e-8997-a7f877a77b58 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:15.403993 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb4805-62f2-496e-8997-a7f877a77b58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:15.404300 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb4805-62f2-496e-8997-a7f877a77b58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:15.405702 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb4805-62f2-496e-8997-a7f877a77b58 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:15.416800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 533/1074] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:15.422668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc24eb-1b82-4e13-854c-e00212313e5e None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:15.530353 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc24eb-1b82-4e13-854c-e00212313e5e None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:15.566529 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e25220-7ee3-4a49-8364-8cbcdc8d3c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:15.567314 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e25220-7ee3-4a49-8364-8cbcdc8d3c72 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:15.567597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e25220-7ee3-4a49-8364-8cbcdc8d3c72 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:15.567870 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e25220-7ee3-4a49-8364-8cbcdc8d3c72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:15.575413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e25220-7ee3-4a49-8364-8cbcdc8d3c72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:15.576869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e25220-7ee3-4a49-8364-8cbcdc8d3c72 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:15.591737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-02e25220-7ee3-4a49-8364-8cbcdc8d3c72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:27:15.591737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:15.591737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:15.591737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:15.591737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:15.591737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:15.591737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:15.591737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:15.591737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:15.591737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:15.591737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:15.591737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:15.591737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:27:15.591737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:27:15.591737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:27:15.592842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:27:15.592842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:15.592842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:15.592842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:15.592842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:15.592842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:15.592842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:15.592842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:15.592842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:15.592842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:15.592842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:15.592842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:15.592842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:15.592842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:15.592842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:15.592842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:15.592842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:15.594140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:27:15.594140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:27:15.594140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:27:15.594140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:27:15.594140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:27:15.594140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:27:15.594140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:27:15.594140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:15.594140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 542/1075] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:27:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:27:15.599202 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb4805-62f2-496e-8997-a7f877a77b58 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:15.677280 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:15.703592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb4805-62f2-496e-8997-a7f877a77b58 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:15.739044 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef771ba-6227-4ca1-aa58-640bac55059a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:15.739721 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef771ba-6227-4ca1-aa58-640bac55059a None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:15.739999 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef771ba-6227-4ca1-aa58-640bac55059a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:15.740268 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef771ba-6227-4ca1-aa58-640bac55059a None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:15.741056 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef771ba-6227-4ca1-aa58-640bac55059a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:15.741385 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef771ba-6227-4ca1-aa58-640bac55059a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:15.742803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef771ba-6227-4ca1-aa58-640bac55059a None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:15.756291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 534/1076] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:27:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:15.763274 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e25220-7ee3-4a49-8364-8cbcdc8d3c72 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:15.797723 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:15.863739 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e25220-7ee3-4a49-8364-8cbcdc8d3c72 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:15.898020 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c530a-84eb-4e03-a2a9-85cff0c45a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:15.898910 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c530a-84eb-4e03-a2a9-85cff0c45a0e None admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:15.899193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c530a-84eb-4e03-a2a9-85cff0c45a0e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:15.899468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c530a-84eb-4e03-a2a9-85cff0c45a0e None admin] PATH_INFO: `/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/6fbc5240f5214aaea4418052b94739e2/roles/4eefa64717a84653822b16532ebf6e75` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:15.901746 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c530a-84eb-4e03-a2a9-85cff0c45a0e None admin] RBAC: Authorizing `identity:create_grant(project_id=37e4afa595894a9994a8e6981ebb7c41, user_id=6fbc5240f5214aaea4418052b94739e2, role_id=4eefa64717a84653822b16532ebf6e75)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:15.903191 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c530a-84eb-4e03-a2a9-85cff0c45a0e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:15.917678 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 543/1077] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:27:15 2022] PUT /identity/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/6fbc5240f5214aaea4418052b94739e2/roles/4eefa64717a84653822b16532ebf6e75 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:27:17.069000 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967c687-cff0-4025-8b4d-46f4e881e9f9 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:17.069673 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967c687-cff0-4025-8b4d-46f4e881e9f9 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:17.069673 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967c687-cff0-4025-8b4d-46f4e881e9f9 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:17.071117 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 535/1078] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:17.076867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a8077-c047-451a-8d58-28a7f75de492 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:17.077357 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a8077-c047-451a-8d58-28a7f75de492 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:17.077460 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a8077-c047-451a-8d58-28a7f75de492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:17.095099 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-a44a8077-c047-451a-8d58-28a7f75de492 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:17.096412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a44a8077-c047-451a-8d58-28a7f75de492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:17.171805 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 544/1079] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:17.195042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967c687-cff0-4025-8b4d-46f4e881e9f9 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:17.297145 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967c687-cff0-4025-8b4d-46f4e881e9f9 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:17.330423 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da5a50-361a-42ba-afb9-1035cc4618a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:17.331130 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da5a50-361a-42ba-afb9-1035cc4618a5 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:17.331439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da5a50-361a-42ba-afb9-1035cc4618a5 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:17.331713 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da5a50-361a-42ba-afb9-1035cc4618a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:17.333659 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:17.342807 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da5a50-361a-42ba-afb9-1035cc4618a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:27:17.350641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-59da5a50-361a-42ba-afb9-1035cc4618a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:27:17.350641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:17.350641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:17.350641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:17.350641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:17.350641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:17.350641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:17.350641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:17.350641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:17.350641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:17.350641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:17.350641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:17.350641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:27:17.350641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:27:17.350641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:27:17.350641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:27:17.351686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:27:17.351686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:27:17.351686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:27:17.351686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:27:17.351686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:17.351686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:17.351686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:17.351686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:17.351686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:17.351686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:17.351686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:17.351686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:17.351686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:17.351686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:17.351686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:17.351686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:17.352619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:17.352619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:17.352619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:17.352619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:17.352619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:27:17.352619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:27:17.352619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:27:17.352619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:27:17.352619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:27:17.352619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:27:17.352619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:27:17.352619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:17.352619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 536/1080] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:27:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:17.358414 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a8077-c047-451a-8d58-28a7f75de492 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:17.383760 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:17.459059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a8077-c047-451a-8d58-28a7f75de492 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:17.496793 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362529a-1e2a-4942-826c-62697f09dd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:17.497487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362529a-1e2a-4942-826c-62697f09dd69 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:17.497768 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362529a-1e2a-4942-826c-62697f09dd69 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:17.498044 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362529a-1e2a-4942-826c-62697f09dd69 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:17.498790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362529a-1e2a-4942-826c-62697f09dd69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:17.499098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362529a-1e2a-4942-826c-62697f09dd69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:17.500482 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362529a-1e2a-4942-826c-62697f09dd69 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:17.509472 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 545/1081] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:27:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:17.515472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da5a50-361a-42ba-afb9-1035cc4618a5 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:17.614992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da5a50-361a-42ba-afb9-1035cc4618a5 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:17.649741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef0619-6092-4125-80f6-97a35658c21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:17.651228 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef0619-6092-4125-80f6-97a35658c21b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:17.651506 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef0619-6092-4125-80f6-97a35658c21b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:17.651782 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef0619-6092-4125-80f6-97a35658c21b None admin] PATH_INFO: `/v3/users/8e9363c137754c108d622f5efca3af81` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:17.652997 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef0619-6092-4125-80f6-97a35658c21b None admin] RBAC: Authorizing `identity:get_user(user_id=8e9363c137754c108d622f5efca3af81)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:17.654463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef0619-6092-4125-80f6-97a35658c21b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:17.656110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 537/1082] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:27:17 2022] GET /identity/v3/users/8e9363c137754c108d622f5efca3af81 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:17.661906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362529a-1e2a-4942-826c-62697f09dd69 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:17.747301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:17.763208 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362529a-1e2a-4942-826c-62697f09dd69 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:17.796101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad03be3-4738-4a7d-9898-6c264d7ccd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:17.796966 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad03be3-4738-4a7d-9898-6c264d7ccd84 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:17.797272 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad03be3-4738-4a7d-9898-6c264d7ccd84 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:17.797582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad03be3-4738-4a7d-9898-6c264d7ccd84 None admin] PATH_INFO: `/v3/projects/886e3d01e51c4c759a9de90e182f61db` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:17.798789 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad03be3-4738-4a7d-9898-6c264d7ccd84 None admin] RBAC: Authorizing `identity:get_project(project_id=886e3d01e51c4c759a9de90e182f61db)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:17.800226 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad03be3-4738-4a7d-9898-6c264d7ccd84 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:17.801973 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 546/1083] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:27:17 2022] GET /identity/v3/projects/886e3d01e51c4c759a9de90e182f61db => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:17.808617 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef0619-6092-4125-80f6-97a35658c21b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:17.870556 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:17.909553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef0619-6092-4125-80f6-97a35658c21b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:17.942980 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cc4a5-4656-4e17-b2d4-a31d97f236a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:17.943879 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cc4a5-4656-4e17-b2d4-a31d97f236a1 None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:17.944160 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cc4a5-4656-4e17-b2d4-a31d97f236a1 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:17.944444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cc4a5-4656-4e17-b2d4-a31d97f236a1 None admin] PATH_INFO: `/v3/projects/886e3d01e51c4c759a9de90e182f61db/users/8e9363c137754c108d622f5efca3af81/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:17.946486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cc4a5-4656-4e17-b2d4-a31d97f236a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=886e3d01e51c4c759a9de90e182f61db, user_id=8e9363c137754c108d622f5efca3af81, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:17.948023 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cc4a5-4656-4e17-b2d4-a31d97f236a1 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:17.949201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b8512-da14-4253-af35-637a09e4f14e None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:17.949484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b8512-da14-4253-af35-637a09e4f14e None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:17.949756 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b8512-da14-4253-af35-637a09e4f14e None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:17.951248 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 547/1084] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:27:17.958187 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5e10f-6728-4781-939d-b3291a442608 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:17.958467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5e10f-6728-4781-939d-b3291a442608 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:17.958738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5e10f-6728-4781-939d-b3291a442608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:17.963908 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 538/1085] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:27:17 2022] PUT /identity/v3/projects/886e3d01e51c4c759a9de90e182f61db/users/8e9363c137754c108d622f5efca3af81/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:27:17.978216 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-97b5e10f-6728-4781-939d-b3291a442608 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:17.980094 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-97b5e10f-6728-4781-939d-b3291a442608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:18.053989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 548/1086] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:18.078300 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cc4a5-4656-4e17-b2d4-a31d97f236a1 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:18.189321 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cc4a5-4656-4e17-b2d4-a31d97f236a1 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:18.212962 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:18.228528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ff6c0-07eb-4b51-9308-3701a6d82fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:18.229408 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ff6c0-07eb-4b51-9308-3701a6d82fcf None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:18.229691 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ff6c0-07eb-4b51-9308-3701a6d82fcf None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:18.229965 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ff6c0-07eb-4b51-9308-3701a6d82fcf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:18.241128 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ff6c0-07eb-4b51-9308-3701a6d82fcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:27:18.249434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-ba2ff6c0-07eb-4b51-9308-3701a6d82fcf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:27:18.249434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:18.249434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:18.249434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:18.249434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:18.249434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:18.249434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:18.249434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:18.249434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:18.249434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:18.249434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:18.249434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:18.249434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:27:18.249434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:27:18.249434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:27:18.249434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:27:18.251505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:27:18.251505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:27:18.251505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:27:18.251505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:27:18.251505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:18.251505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:18.251505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:18.251505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:18.251505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:18.251505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:18.251505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:18.251505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:18.251505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:18.251505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:18.251505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:18.251505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:18.253427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:18.253427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:18.253427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:18.253427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:18.253427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:27:18.253427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:27:18.253427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:27:18.253427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:27:18.253427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:27:18.253427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:27:18.253427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:27:18.253427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:18.253427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 539/1087] 10.222.0.71 () {56 vars in 1120 bytes} [Thu Jul 21 20:27:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:18.256786 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5e10f-6728-4781-939d-b3291a442608 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:18.347312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:18.364860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5e10f-6728-4781-939d-b3291a442608 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:18.399150 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e41c2-88e8-4859-8e23-a3b2565999fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:18.399810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e41c2-88e8-4859-8e23-a3b2565999fd None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:18.400130 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e41c2-88e8-4859-8e23-a3b2565999fd None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:18.400359 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e41c2-88e8-4859-8e23-a3b2565999fd None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:18.401700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e41c2-88e8-4859-8e23-a3b2565999fd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:18.402001 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e41c2-88e8-4859-8e23-a3b2565999fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:18.403372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e41c2-88e8-4859-8e23-a3b2565999fd None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:18.411629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 549/1088] 10.222.0.71 () {56 vars in 1115 bytes} [Thu Jul 21 20:27:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:18.418840 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ff6c0-07eb-4b51-9308-3701a6d82fcf None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:18.526864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ff6c0-07eb-4b51-9308-3701a6d82fcf None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:18.560556 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7ebc5-9edd-45b4-870d-95f0e9999550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:18.561388 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7ebc5-9edd-45b4-870d-95f0e9999550 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:18.561675 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7ebc5-9edd-45b4-870d-95f0e9999550 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:18.561950 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7ebc5-9edd-45b4-870d-95f0e9999550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:18.573570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7ebc5-9edd-45b4-870d-95f0e9999550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:18.575030 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7ebc5-9edd-45b4-870d-95f0e9999550 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:18.592432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-0fd7ebc5-9edd-45b4-870d-95f0e9999550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:27:18.592432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:18.592432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:18.592432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:18.592432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:18.592432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:18.592432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:18.592432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:18.592432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:18.592432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:18.592432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:18.592432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:18.592432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:27:18.592432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:27:18.592432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:27:18.593520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:27:18.593520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:18.593520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:18.593520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:18.593520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:18.593520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:18.593520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:18.593520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:18.593520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:18.593520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:18.593520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:18.593520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:18.593520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:18.593520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:18.593520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:18.593520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:18.593520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:18.594723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:27:18.594723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:27:18.594723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:27:18.594723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:18.594723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 540/1089] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:27:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:18.600580 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e41c2-88e8-4859-8e23-a3b2565999fd None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:18.705422 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e41c2-88e8-4859-8e23-a3b2565999fd None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:18.739993 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa0b3c-e396-4eb9-8fbc-93a0ffb9e713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:18.740653 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa0b3c-e396-4eb9-8fbc-93a0ffb9e713 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:18.740926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa0b3c-e396-4eb9-8fbc-93a0ffb9e713 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:18.741228 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa0b3c-e396-4eb9-8fbc-93a0ffb9e713 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:18.741943 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa0b3c-e396-4eb9-8fbc-93a0ffb9e713 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:18.742247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa0b3c-e396-4eb9-8fbc-93a0ffb9e713 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:18.743607 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa0b3c-e396-4eb9-8fbc-93a0ffb9e713 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:18.755916 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 550/1090] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:18.762627 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7ebc5-9edd-45b4-870d-95f0e9999550 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:18.799424 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:18.866782 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7ebc5-9edd-45b4-870d-95f0e9999550 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:18.901201 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3f538-1f27-4090-9785-8785f43a1353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:18.901996 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3f538-1f27-4090-9785-8785f43a1353 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:18.902273 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3f538-1f27-4090-9785-8785f43a1353 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:18.902541 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3f538-1f27-4090-9785-8785f43a1353 None admin] PATH_INFO: `/v3/users/nova` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:18.910576 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3f538-1f27-4090-9785-8785f43a1353 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:18.912017 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3f538-1f27-4090-9785-8785f43a1353 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:18.926492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-2ba3f538-1f27-4090-9785-8785f43a1353 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:27:18.926492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:18.926492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:18.926492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:18.926492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:18.926492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:18.926492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:18.926492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:18.926492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:18.926492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:18.926492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:18.926492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:18.926492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:27:18.926492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:27:18.926492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:27:18.926492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:27:18.928447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:18.928447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:18.928447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:27:18.928447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:27:18.928447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:27:18.928447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:27:18.928447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:18.928447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:18.928447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:18.928447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:18.928447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:18.928447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:18.928447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:18.928447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:18.928447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:18.928447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:18.928447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:18.928447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:18.929771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:18.929771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:18.929771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:27:18.929771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:27:18.929771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:27:18.929771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:27:18.929771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:27:18.929771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:27:18.929771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:27:18.929771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:18.929771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 541/1091] 10.222.0.71 () {56 vars in 1093 bytes} [Thu Jul 21 20:27:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:18.934394 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa0b3c-e396-4eb9-8fbc-93a0ffb9e713 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:18.943715 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:19.050317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa0b3c-e396-4eb9-8fbc-93a0ffb9e713 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:19.089691 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de896510-7c94-4c48-bbb0-32d907dd348a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:19.090377 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de896510-7c94-4c48-bbb0-32d907dd348a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:19.090656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de896510-7c94-4c48-bbb0-32d907dd348a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:19.090927 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de896510-7c94-4c48-bbb0-32d907dd348a None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:19.091807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de896510-7c94-4c48-bbb0-32d907dd348a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:19.092115 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de896510-7c94-4c48-bbb0-32d907dd348a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:19.093663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de896510-7c94-4c48-bbb0-32d907dd348a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:19.115034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 551/1092] 10.222.0.71 () {56 vars in 1133 bytes} [Thu Jul 21 20:27:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:19.121520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3f538-1f27-4090-9785-8785f43a1353 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:19.222468 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:19.230548 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3f538-1f27-4090-9785-8785f43a1353 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:19.266497 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4090cbe-cc6f-489c-8ad9-11bf999e89f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:19.267151 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4090cbe-cc6f-489c-8ad9-11bf999e89f3 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:19.267432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4090cbe-cc6f-489c-8ad9-11bf999e89f3 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:19.267701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4090cbe-cc6f-489c-8ad9-11bf999e89f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:19.338682 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4090cbe-cc6f-489c-8ad9-11bf999e89f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:19.340048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4090cbe-cc6f-489c-8ad9-11bf999e89f3 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:19.438450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:19.449797 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 542/1093] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:27:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:27:19.456198 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de896510-7c94-4c48-bbb0-32d907dd348a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:19.541055 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:19.587841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de896510-7c94-4c48-bbb0-32d907dd348a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:19.628442 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be218ed-757a-47a2-9302-a782fb93501c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:19.629224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be218ed-757a-47a2-9302-a782fb93501c None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:19.629509 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be218ed-757a-47a2-9302-a782fb93501c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:19.629788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be218ed-757a-47a2-9302-a782fb93501c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:19.642075 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be218ed-757a-47a2-9302-a782fb93501c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:19.643608 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be218ed-757a-47a2-9302-a782fb93501c None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:19.663836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-6be218ed-757a-47a2-9302-a782fb93501c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:27:19.663836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:19.663836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:19.663836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:19.663836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:19.663836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:19.663836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:19.663836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:19.663836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:19.663836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:19.663836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:19.663836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:19.663836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:27:19.663836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:27:19.663836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:27:19.665869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:27:19.665869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:19.665869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:19.665869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:19.665869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:19.665869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:19.665869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:19.665869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:19.665869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:19.665869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:19.665869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:19.665869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:19.665869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:19.665869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:19.665869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:19.665869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:19.665869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:19.668386 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:27:19.668386 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:27:19.668386 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:27:19.668386 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:19.668386 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 552/1094] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:27:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:19.673000 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4090cbe-cc6f-489c-8ad9-11bf999e89f3 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:19.787496 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4090cbe-cc6f-489c-8ad9-11bf999e89f3 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:19.825825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cff225-c27f-4862-b7c4-660ad515630b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:19.826519 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cff225-c27f-4862-b7c4-660ad515630b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:19.826780 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cff225-c27f-4862-b7c4-660ad515630b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:19.827055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cff225-c27f-4862-b7c4-660ad515630b None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:19.827811 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cff225-c27f-4862-b7c4-660ad515630b None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:19.828086 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cff225-c27f-4862-b7c4-660ad515630b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:19.829560 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cff225-c27f-4862-b7c4-660ad515630b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:19.843564 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 543/1095] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:19.849646 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be218ed-757a-47a2-9302-a782fb93501c None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:19.961327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be218ed-757a-47a2-9302-a782fb93501c None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:19.978409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:19.997307 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd96340-1e46-402e-8116-b0b4e0f95582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:19.998004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd96340-1e46-402e-8116-b0b4e0f95582 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:19.998286 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd96340-1e46-402e-8116-b0b4e0f95582 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:19.998564 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd96340-1e46-402e-8116-b0b4e0f95582 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:20.005950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd96340-1e46-402e-8116-b0b4e0f95582 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:20.007408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd96340-1e46-402e-8116-b0b4e0f95582 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:20.022069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-6dd96340-1e46-402e-8116-b0b4e0f95582 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:27:20.022069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:20.022069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:20.022069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:20.022069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:20.022069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:20.022069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:20.022069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:20.022069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:20.022069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:20.022069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:20.022069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:20.022069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:27:20.022069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:27:20.022069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:27:20.023056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:27:20.023056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:20.023056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:20.023056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:20.023056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:20.023056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:20.023056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:20.023056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:20.023056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:20.023056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:20.023056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:20.023056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:20.023056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:20.023056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:20.023056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:20.023056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:20.023056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:20.024365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:27:20.024365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:27:20.024365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:27:20.024365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:27:20.024365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:27:20.024365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:27:20.024365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:27:20.024365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:20.024365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 553/1096] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:27:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:20.029288 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cff225-c27f-4862-b7c4-660ad515630b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:20.051204 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577f21d-d143-4092-8d9d-89df69bdfa8f None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:20.051487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577f21d-d143-4092-8d9d-89df69bdfa8f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:20.051769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577f21d-d143-4092-8d9d-89df69bdfa8f None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:20.053196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 554/1097] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:27:20.059046 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac48a8-27e2-434c-944a-8bdaa612e7a4 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:20.059335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac48a8-27e2-434c-944a-8bdaa612e7a4 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:20.059612 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac48a8-27e2-434c-944a-8bdaa612e7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:20.076900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-bbac48a8-27e2-434c-944a-8bdaa612e7a4 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:20.078132 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-bbac48a8-27e2-434c-944a-8bdaa612e7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:20.088666 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:20.138012 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cff225-c27f-4862-b7c4-660ad515630b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:20.150399 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 555/1098] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:27:20.173977 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac48a8-27e2-434c-944a-8bdaa612e7a4 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:20.174224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d4417-fa2a-4035-b405-b73ef189185c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:20.174546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d4417-fa2a-4035-b405-b73ef189185c None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:20.174830 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d4417-fa2a-4035-b405-b73ef189185c None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:20.175109 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d4417-fa2a-4035-b405-b73ef189185c None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:20.175947 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d4417-fa2a-4035-b405-b73ef189185c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:20.176270 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d4417-fa2a-4035-b405-b73ef189185c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:20.177747 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d4417-fa2a-4035-b405-b73ef189185c None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:20.191805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 544/1099] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:27:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:20.198959 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d4417-fa2a-4035-b405-b73ef189185c None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:20.289732 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac48a8-27e2-434c-944a-8bdaa612e7a4 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:20.298199 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:20.317092 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d4417-fa2a-4035-b405-b73ef189185c None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:20.333494 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:20.341152 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9744f-3b06-465e-a438-372337228ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:20.341948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9744f-3b06-465e-a438-372337228ee5 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:20.342227 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9744f-3b06-465e-a438-372337228ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:20.342506 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9744f-3b06-465e-a438-372337228ee5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:20.354038 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9744f-3b06-465e-a438-372337228ee5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:27:20.356443 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be90064-8e2d-4fdb-9291-b24e2399421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:20.357123 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be90064-8e2d-4fdb-9291-b24e2399421d None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:20.357412 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be90064-8e2d-4fdb-9291-b24e2399421d None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:20.357690 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be90064-8e2d-4fdb-9291-b24e2399421d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:20.358544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be90064-8e2d-4fdb-9291-b24e2399421d None admin] RBAC: Adding query filter params (role.id=4eefa64717a84653822b16532ebf6e75, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=6fbc5240f5214aaea4418052b94739e2) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:20.358866 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be90064-8e2d-4fdb-9291-b24e2399421d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:20.360314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be90064-8e2d-4fdb-9291-b24e2399421d None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:20.361699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-e8a9744f-3b06-465e-a438-372337228ee5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:27:20.361699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:20.361699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:20.361699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:20.361699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:20.361699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:20.361699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:20.361699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:20.361699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:20.361699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:20.361699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:20.361699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:20.361699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:27:20.361699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:27:20.361699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:27:20.361699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:27:20.362645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:27:20.362645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:27:20.362645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:27:20.362645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:27:20.362645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:20.362645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:20.362645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:20.362645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:20.362645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:20.362645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:20.362645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:20.362645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:20.362645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:20.362645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:20.362645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:20.362645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:20.363693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:20.363693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:20.363693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:20.363693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:20.363693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:27:20.363693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:27:20.363693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:27:20.363693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:27:20.363693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:27:20.363693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:27:20.363693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:27:20.363693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:20.363693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 556/1100] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:27:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:27:20.367213 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 545/1101] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:27:20 2022] GET /identity/v3/role_assignments?user.id=6fbc5240f5214aaea4418052b94739e2&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=4eefa64717a84653822b16532ebf6e75 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:20.369044 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9744f-3b06-465e-a438-372337228ee5 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:20.517490 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9744f-3b06-465e-a438-372337228ee5 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:20.554257 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe7c4d-0bdc-4bb3-b6f8-01ccc82a0396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:20.554851 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe7c4d-0bdc-4bb3-b6f8-01ccc82a0396 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:20.555128 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe7c4d-0bdc-4bb3-b6f8-01ccc82a0396 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:20.555399 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe7c4d-0bdc-4bb3-b6f8-01ccc82a0396 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:20.556149 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe7c4d-0bdc-4bb3-b6f8-01ccc82a0396 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:20.556454 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe7c4d-0bdc-4bb3-b6f8-01ccc82a0396 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:20.557873 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe7c4d-0bdc-4bb3-b6f8-01ccc82a0396 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:20.567984 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 557/1102] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:27:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:20.574623 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be90064-8e2d-4fdb-9291-b24e2399421d None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:20.679396 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be90064-8e2d-4fdb-9291-b24e2399421d None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:20.719560 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1b56f-3adc-41ce-a519-7d7f636aa815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:20.720461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1b56f-3adc-41ce-a519-7d7f636aa815 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:20.720744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1b56f-3adc-41ce-a519-7d7f636aa815 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:20.721019 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1b56f-3adc-41ce-a519-7d7f636aa815 None admin] PATH_INFO: `/v3/users/8e9363c137754c108d622f5efca3af81` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:20.722300 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1b56f-3adc-41ce-a519-7d7f636aa815 None admin] RBAC: Authorizing `identity:get_user(user_id=8e9363c137754c108d622f5efca3af81)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:20.723980 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1b56f-3adc-41ce-a519-7d7f636aa815 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:20.725667 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 546/1103] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:27:20 2022] GET /identity/v3/users/8e9363c137754c108d622f5efca3af81 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:20.731621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe7c4d-0bdc-4bb3-b6f8-01ccc82a0396 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:20.763048 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:20.836927 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe7c4d-0bdc-4bb3-b6f8-01ccc82a0396 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:20.871111 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37327c10-87af-4daf-9706-1719ee53a85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:20.871750 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37327c10-87af-4daf-9706-1719ee53a85c None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:20.872033 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37327c10-87af-4daf-9706-1719ee53a85c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:20.872301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37327c10-87af-4daf-9706-1719ee53a85c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:20.955759 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37327c10-87af-4daf-9706-1719ee53a85c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:20.955759 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37327c10-87af-4daf-9706-1719ee53a85c None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:20.988870 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:21.064370 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 558/1104] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:27:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:27:21.070417 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1b56f-3adc-41ce-a519-7d7f636aa815 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:21.122332 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:21.179289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1b56f-3adc-41ce-a519-7d7f636aa815 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:21.216001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e6e37-afb4-4cbe-9c5b-f3e1129437c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:21.216870 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e6e37-afb4-4cbe-9c5b-f3e1129437c3 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:21.217190 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e6e37-afb4-4cbe-9c5b-f3e1129437c3 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:21.217469 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e6e37-afb4-4cbe-9c5b-f3e1129437c3 None admin] PATH_INFO: `/v3/projects/886e3d01e51c4c759a9de90e182f61db` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:21.218674 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e6e37-afb4-4cbe-9c5b-f3e1129437c3 None admin] RBAC: Authorizing `identity:get_project(project_id=886e3d01e51c4c759a9de90e182f61db)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:21.220110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e6e37-afb4-4cbe-9c5b-f3e1129437c3 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:21.221893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 547/1105] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:27:21 2022] GET /identity/v3/projects/886e3d01e51c4c759a9de90e182f61db => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:21.228668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37327c10-87af-4daf-9706-1719ee53a85c None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:21.346614 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37327c10-87af-4daf-9706-1719ee53a85c None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:21.373606 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:21.382165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa96f1-d431-44b2-8d32-4a4ca58334a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:21.382822 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa96f1-d431-44b2-8d32-4a4ca58334a2 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:21.383104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa96f1-d431-44b2-8d32-4a4ca58334a2 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:21.383381 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa96f1-d431-44b2-8d32-4a4ca58334a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:21.384223 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa96f1-d431-44b2-8d32-4a4ca58334a2 None admin] RBAC: Adding query filter params (role.id=19f5e72c0be44a1197c7288de82e79e8, scope.project.id=886e3d01e51c4c759a9de90e182f61db, user.id=8e9363c137754c108d622f5efca3af81) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:21.384538 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa96f1-d431-44b2-8d32-4a4ca58334a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:21.386537 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa96f1-d431-44b2-8d32-4a4ca58334a2 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:21.392769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 559/1106] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:27:21 2022] GET /identity/v3/role_assignments?user.id=8e9363c137754c108d622f5efca3af81&scope.project.id=886e3d01e51c4c759a9de90e182f61db&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:23.462512 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7e3b1-4a6c-4a95-8829-872958024ee8 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:23.463517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7e3b1-4a6c-4a95-8829-872958024ee8 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:23.463691 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7e3b1-4a6c-4a95-8829-872958024ee8 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:23.465100 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 548/1107] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:23.470653 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad521e-15d0-4e6f-a5d0-bc2a8a250558 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:23.471103 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad521e-15d0-4e6f-a5d0-bc2a8a250558 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:23.471204 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad521e-15d0-4e6f-a5d0-bc2a8a250558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:23.488985 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-f9ad521e-15d0-4e6f-a5d0-bc2a8a250558 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:23.490219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f9ad521e-15d0-4e6f-a5d0-bc2a8a250558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:23.557417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 560/1108] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:23.582506 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7e3b1-4a6c-4a95-8829-872958024ee8 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:23.686352 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7e3b1-4a6c-4a95-8829-872958024ee8 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:23.700675 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:23.721004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158d2a3-b106-4a76-a2b8-8733cdc4c6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:23.721758 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158d2a3-b106-4a76-a2b8-8733cdc4c6f6 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:23.722041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158d2a3-b106-4a76-a2b8-8733cdc4c6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:23.722312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158d2a3-b106-4a76-a2b8-8733cdc4c6f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:23.723759 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158d2a3-b106-4a76-a2b8-8733cdc4c6f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:23.725221 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158d2a3-b106-4a76-a2b8-8733cdc4c6f6 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:23.726880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 549/1109] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:27:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:23.732764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad521e-15d0-4e6f-a5d0-bc2a8a250558 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:23.832556 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad521e-15d0-4e6f-a5d0-bc2a8a250558 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:23.855205 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:23.867284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67750c7-c637-4906-a79d-0481088af567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:23.867956 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67750c7-c637-4906-a79d-0481088af567 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:23.868240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67750c7-c637-4906-a79d-0481088af567 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:23.868517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67750c7-c637-4906-a79d-0481088af567 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:23.869781 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67750c7-c637-4906-a79d-0481088af567 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:23.871229 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67750c7-c637-4906-a79d-0481088af567 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:23.894251 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 561/1110] 10.222.0.71 () {60 vars in 1132 bytes} [Thu Jul 21 20:27:23 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:27:26.033696 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712db611-fdfb-43e2-911f-a74d62b4a6a1 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:26.034524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712db611-fdfb-43e2-911f-a74d62b4a6a1 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:26.034524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712db611-fdfb-43e2-911f-a74d62b4a6a1 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:26.035645 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 550/1111] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:27:26.041793 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d67d0-879d-4682-a0d0-4862e20cfb21 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:26.042636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d67d0-879d-4682-a0d0-4862e20cfb21 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:26.042636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d67d0-879d-4682-a0d0-4862e20cfb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:26.060280 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-f16d67d0-879d-4682-a0d0-4862e20cfb21 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:26.061543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f16d67d0-879d-4682-a0d0-4862e20cfb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:26.135846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 562/1112] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:27:26.160088 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712db611-fdfb-43e2-911f-a74d62b4a6a1 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:26.265647 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712db611-fdfb-43e2-911f-a74d62b4a6a1 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:26.302851 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696dbab-3711-4e82-88bb-681980cf4cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:26.303715 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696dbab-3711-4e82-88bb-681980cf4cfa None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:26.304051 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696dbab-3711-4e82-88bb-681980cf4cfa None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:26.304497 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696dbab-3711-4e82-88bb-681980cf4cfa None admin] PATH_INFO: `/v3/roles/bad7164dd96e4048bab9360ef3f6769e` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:26.306992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696dbab-3711-4e82-88bb-681980cf4cfa None admin] RBAC: Authorizing `identity:get_role(role_id=bad7164dd96e4048bab9360ef3f6769e)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:26.308662 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696dbab-3711-4e82-88bb-681980cf4cfa None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:26.310152 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 551/1113] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:27:26 2022] GET /identity/v3/roles/bad7164dd96e4048bab9360ef3f6769e => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:26.316223 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d67d0-879d-4682-a0d0-4862e20cfb21 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:26.418984 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d67d0-879d-4682-a0d0-4862e20cfb21 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:26.429713 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:26.457838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05421a5a-729b-4290-9e5a-28e71d23e620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:26.458726 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05421a5a-729b-4290-9e5a-28e71d23e620 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:26.459006 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05421a5a-729b-4290-9e5a-28e71d23e620 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:26.459287 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05421a5a-729b-4290-9e5a-28e71d23e620 None admin] PATH_INFO: `/v3/users/dcdae6ff043c4da395c770e5583e3c56` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:26.474471 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05421a5a-729b-4290-9e5a-28e71d23e620 None admin] RBAC: Authorizing `identity:get_user(user_id=dcdae6ff043c4da395c770e5583e3c56)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:26.475950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05421a5a-729b-4290-9e5a-28e71d23e620 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:26.477737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 563/1114] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:27:26 2022] GET /identity/v3/users/dcdae6ff043c4da395c770e5583e3c56 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:26.484189 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696dbab-3711-4e82-88bb-681980cf4cfa None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:26.536378 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:26.591404 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696dbab-3711-4e82-88bb-681980cf4cfa None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:26.625959 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc84e00-77c1-4c73-bcc6-1a60dcbbcc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:26.626600 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc84e00-77c1-4c73-bcc6-1a60dcbbcc14 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:26.626880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc84e00-77c1-4c73-bcc6-1a60dcbbcc14 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:26.627150 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc84e00-77c1-4c73-bcc6-1a60dcbbcc14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:26.699093 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc84e00-77c1-4c73-bcc6-1a60dcbbcc14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:26.700550 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc84e00-77c1-4c73-bcc6-1a60dcbbcc14 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:26.752197 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:26.807894 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 552/1115] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:27:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:27:26.814262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05421a5a-729b-4290-9e5a-28e71d23e620 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:26.918458 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05421a5a-729b-4290-9e5a-28e71d23e620 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:26.954615 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a135b-66b3-4427-8ab5-ca3f9f4c5b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:26.955483 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a135b-66b3-4427-8ab5-ca3f9f4c5b46 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:26.955766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a135b-66b3-4427-8ab5-ca3f9f4c5b46 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:26.956038 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a135b-66b3-4427-8ab5-ca3f9f4c5b46 None admin] PATH_INFO: `/v3/projects/886e3d01e51c4c759a9de90e182f61db` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:26.957272 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a135b-66b3-4427-8ab5-ca3f9f4c5b46 None admin] RBAC: Authorizing `identity:get_project(project_id=886e3d01e51c4c759a9de90e182f61db)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:26.958706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a135b-66b3-4427-8ab5-ca3f9f4c5b46 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:26.960430 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 564/1116] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:27:26 2022] GET /identity/v3/projects/886e3d01e51c4c759a9de90e182f61db => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:26.967335 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc84e00-77c1-4c73-bcc6-1a60dcbbcc14 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:27.074738 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc84e00-77c1-4c73-bcc6-1a60dcbbcc14 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:27.111967 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4ff91-a68a-4b2f-9924-352550a4b40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:27.112623 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4ff91-a68a-4b2f-9924-352550a4b40f None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:27.112909 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4ff91-a68a-4b2f-9924-352550a4b40f None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:27.113231 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4ff91-a68a-4b2f-9924-352550a4b40f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:27.114083 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4ff91-a68a-4b2f-9924-352550a4b40f None admin] RBAC: Adding query filter params (role.id=bad7164dd96e4048bab9360ef3f6769e, scope.project.id=886e3d01e51c4c759a9de90e182f61db, user.id=dcdae6ff043c4da395c770e5583e3c56) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:27.114399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4ff91-a68a-4b2f-9924-352550a4b40f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:27.115851 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4ff91-a68a-4b2f-9924-352550a4b40f None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:27.121692 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 553/1117] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:27:26 2022] GET /identity/v3/role_assignments?user.id=dcdae6ff043c4da395c770e5583e3c56&scope.project.id=886e3d01e51c4c759a9de90e182f61db&role.id=bad7164dd96e4048bab9360ef3f6769e => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:29.190699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1282db-6efc-4a35-9191-57ce408ec5bf None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:29.191260 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1282db-6efc-4a35-9191-57ce408ec5bf None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:29.191260 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1282db-6efc-4a35-9191-57ce408ec5bf None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:29.192654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 565/1118] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:29.198692 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f7c16-a56e-407f-a33f-0abd6ccdf61f None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:29.199421 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f7c16-a56e-407f-a33f-0abd6ccdf61f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:29.199421 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f7c16-a56e-407f-a33f-0abd6ccdf61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:29.215865 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-ff0f7c16-a56e-407f-a33f-0abd6ccdf61f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:29.217082 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-ff0f7c16-a56e-407f-a33f-0abd6ccdf61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:29.227201 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:29.286753 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 554/1119] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:29.310965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1282db-6efc-4a35-9191-57ce408ec5bf None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:29.348555 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:29.419611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1282db-6efc-4a35-9191-57ce408ec5bf None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:29.455138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b540e2c-26ff-4e59-8c70-280ac9dfe06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:29.456009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b540e2c-26ff-4e59-8c70-280ac9dfe06b None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:29.456289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b540e2c-26ff-4e59-8c70-280ac9dfe06b None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:29.456562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b540e2c-26ff-4e59-8c70-280ac9dfe06b None admin] PATH_INFO: `/v3/roles/bad7164dd96e4048bab9360ef3f6769e` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:29.458074 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b540e2c-26ff-4e59-8c70-280ac9dfe06b None admin] RBAC: Authorizing `identity:get_role(role_id=bad7164dd96e4048bab9360ef3f6769e)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:29.459454 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b540e2c-26ff-4e59-8c70-280ac9dfe06b None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:29.460839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 566/1120] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:27:29 2022] GET /identity/v3/roles/bad7164dd96e4048bab9360ef3f6769e => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:29.466846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f7c16-a56e-407f-a33f-0abd6ccdf61f None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:29.573726 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f7c16-a56e-407f-a33f-0abd6ccdf61f None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:29.614717 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51692496-0170-4d79-b00e-428b2d5987b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:29.615596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51692496-0170-4d79-b00e-428b2d5987b9 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:29.615881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51692496-0170-4d79-b00e-428b2d5987b9 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:29.616159 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51692496-0170-4d79-b00e-428b2d5987b9 None admin] PATH_INFO: `/v3/users/dcdae6ff043c4da395c770e5583e3c56` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:29.617410 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51692496-0170-4d79-b00e-428b2d5987b9 None admin] RBAC: Authorizing `identity:get_user(user_id=dcdae6ff043c4da395c770e5583e3c56)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:29.619413 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51692496-0170-4d79-b00e-428b2d5987b9 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:29.621086 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 555/1121] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:27:29 2022] GET /identity/v3/users/dcdae6ff043c4da395c770e5583e3c56 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:29.627114 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b540e2c-26ff-4e59-8c70-280ac9dfe06b None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:29.737279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b540e2c-26ff-4e59-8c70-280ac9dfe06b None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:29.738824 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:29.783401 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc172a-6184-4aa9-9c89-6cce71764432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:29.784269 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc172a-6184-4aa9-9c89-6cce71764432 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:29.784548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc172a-6184-4aa9-9c89-6cce71764432 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:29.784812 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc172a-6184-4aa9-9c89-6cce71764432 None admin] PATH_INFO: `/v3/projects/886e3d01e51c4c759a9de90e182f61db` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:29.786043 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc172a-6184-4aa9-9c89-6cce71764432 None admin] RBAC: Authorizing `identity:get_project(project_id=886e3d01e51c4c759a9de90e182f61db)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:29.787470 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc172a-6184-4aa9-9c89-6cce71764432 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:29.789186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 567/1122] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:27:29 2022] GET /identity/v3/projects/886e3d01e51c4c759a9de90e182f61db => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:29.796574 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51692496-0170-4d79-b00e-428b2d5987b9 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:29.811796 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:29.912231 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51692496-0170-4d79-b00e-428b2d5987b9 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:29.969030 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c213358-01cb-4129-936c-94e593d0e3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:29.970469 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c213358-01cb-4129-936c-94e593d0e3c8 None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:29.970927 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c213358-01cb-4129-936c-94e593d0e3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:29.971416 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c213358-01cb-4129-936c-94e593d0e3c8 None admin] PATH_INFO: `/v3/projects/886e3d01e51c4c759a9de90e182f61db/users/dcdae6ff043c4da395c770e5583e3c56/roles/bad7164dd96e4048bab9360ef3f6769e` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:29.974168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c213358-01cb-4129-936c-94e593d0e3c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=886e3d01e51c4c759a9de90e182f61db, user_id=dcdae6ff043c4da395c770e5583e3c56, role_id=bad7164dd96e4048bab9360ef3f6769e)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:29.975636 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c213358-01cb-4129-936c-94e593d0e3c8 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:29.991105 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 556/1123] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:27:29 2022] PUT /identity/v3/projects/886e3d01e51c4c759a9de90e182f61db/users/dcdae6ff043c4da395c770e5583e3c56/roles/bad7164dd96e4048bab9360ef3f6769e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:27:32.102218 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd58a4d-1dcd-44ef-a31c-ef558acf59a0 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:32.103108 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd58a4d-1dcd-44ef-a31c-ef558acf59a0 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:32.103108 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd58a4d-1dcd-44ef-a31c-ef558acf59a0 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:32.104684 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 568/1124] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:27:32.111547 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6472c28-1bbc-47fd-8c18-ec771e553a2c None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:32.112096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6472c28-1bbc-47fd-8c18-ec771e553a2c None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:32.112519 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6472c28-1bbc-47fd-8c18-ec771e553a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:32.141157 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-d6472c28-1bbc-47fd-8c18-ec771e553a2c None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:32.142980 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d6472c28-1bbc-47fd-8c18-ec771e553a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:32.261526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 557/1125] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:32.285503 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd58a4d-1dcd-44ef-a31c-ef558acf59a0 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:32.404740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd58a4d-1dcd-44ef-a31c-ef558acf59a0 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:32.445415 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198276f0-c0e9-4a69-9556-822222df6bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:32.446296 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198276f0-c0e9-4a69-9556-822222df6bc9 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:32.446575 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198276f0-c0e9-4a69-9556-822222df6bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:32.446847 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198276f0-c0e9-4a69-9556-822222df6bc9 None admin] PATH_INFO: `/v3/roles/bad7164dd96e4048bab9360ef3f6769e` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:32.448484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198276f0-c0e9-4a69-9556-822222df6bc9 None admin] RBAC: Authorizing `identity:get_role(role_id=bad7164dd96e4048bab9360ef3f6769e)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:32.449969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198276f0-c0e9-4a69-9556-822222df6bc9 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:32.451682 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 569/1126] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:27:32 2022] GET /identity/v3/roles/bad7164dd96e4048bab9360ef3f6769e => generated 223 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:32.457957 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6472c28-1bbc-47fd-8c18-ec771e553a2c None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:32.459929 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:32.594749 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6472c28-1bbc-47fd-8c18-ec771e553a2c None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:32.630352 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f957306-bbb5-4cd7-9896-81ad5a0d1ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:32.631232 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f957306-bbb5-4cd7-9896-81ad5a0d1ef5 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:32.631506 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f957306-bbb5-4cd7-9896-81ad5a0d1ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:32.631777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f957306-bbb5-4cd7-9896-81ad5a0d1ef5 None admin] PATH_INFO: `/v3/users/dcdae6ff043c4da395c770e5583e3c56` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:32.632992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f957306-bbb5-4cd7-9896-81ad5a0d1ef5 None admin] RBAC: Authorizing `identity:get_user(user_id=dcdae6ff043c4da395c770e5583e3c56)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:32.634488 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f957306-bbb5-4cd7-9896-81ad5a0d1ef5 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:32.636153 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 558/1127] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:27:32 2022] GET /identity/v3/users/dcdae6ff043c4da395c770e5583e3c56 => generated 286 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:32.642180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198276f0-c0e9-4a69-9556-822222df6bc9 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:32.684442 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:32.750092 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198276f0-c0e9-4a69-9556-822222df6bc9 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:32.786082 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e7e01-03b3-458c-b742-b0394fe2a22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:32.786727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e7e01-03b3-458c-b742-b0394fe2a22a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:32.786998 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e7e01-03b3-458c-b742-b0394fe2a22a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:32.787267 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e7e01-03b3-458c-b742-b0394fe2a22a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:32.856639 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e7e01-03b3-458c-b742-b0394fe2a22a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:32.858125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e7e01-03b3-458c-b742-b0394fe2a22a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:32.898210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:32.963519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 570/1128] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:27:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:27:32.969404 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f957306-bbb5-4cd7-9896-81ad5a0d1ef5 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:33.048971 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:33.082432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f957306-bbb5-4cd7-9896-81ad5a0d1ef5 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:33.119540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e43c7-315e-4597-be9e-dc91dcd29a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:33.120404 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e43c7-315e-4597-be9e-dc91dcd29a98 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:33.120687 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e43c7-315e-4597-be9e-dc91dcd29a98 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:33.120961 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e43c7-315e-4597-be9e-dc91dcd29a98 None admin] PATH_INFO: `/v3/projects/886e3d01e51c4c759a9de90e182f61db` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:33.122204 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e43c7-315e-4597-be9e-dc91dcd29a98 None admin] RBAC: Authorizing `identity:get_project(project_id=886e3d01e51c4c759a9de90e182f61db)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:33.123657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e43c7-315e-4597-be9e-dc91dcd29a98 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:33.125439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 559/1129] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:27:32 2022] GET /identity/v3/projects/886e3d01e51c4c759a9de90e182f61db => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:33.132587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e7e01-03b3-458c-b742-b0394fe2a22a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:33.249059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e7e01-03b3-458c-b742-b0394fe2a22a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:33.282708 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9192a5c-8307-4941-afe1-6dabfe514bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:33.283367 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9192a5c-8307-4941-afe1-6dabfe514bcb None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:33.283647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9192a5c-8307-4941-afe1-6dabfe514bcb None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:33.283922 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9192a5c-8307-4941-afe1-6dabfe514bcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:33.284762 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9192a5c-8307-4941-afe1-6dabfe514bcb None admin] RBAC: Adding query filter params (role.id=bad7164dd96e4048bab9360ef3f6769e, scope.project.id=886e3d01e51c4c759a9de90e182f61db, user.id=dcdae6ff043c4da395c770e5583e3c56) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:33.285120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9192a5c-8307-4941-afe1-6dabfe514bcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:33.286559 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9192a5c-8307-4941-afe1-6dabfe514bcb None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:33.293670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 571/1130] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:27:33 2022] GET /identity/v3/role_assignments?user.id=dcdae6ff043c4da395c770e5583e3c56&scope.project.id=886e3d01e51c4c759a9de90e182f61db&role.id=bad7164dd96e4048bab9360ef3f6769e => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:35.442600 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fabaa4-ee1c-4d78-aad2-ac129e35969f None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:35.443223 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fabaa4-ee1c-4d78-aad2-ac129e35969f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:35.443223 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fabaa4-ee1c-4d78-aad2-ac129e35969f None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:35.444547 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 560/1131] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:35.450603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce291b26-7ed3-4c9f-95bf-fab2bc22eb40 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:35.451179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce291b26-7ed3-4c9f-95bf-fab2bc22eb40 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:35.451313 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce291b26-7ed3-4c9f-95bf-fab2bc22eb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:35.456620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:35.470717 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-ce291b26-7ed3-4c9f-95bf-fab2bc22eb40 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:35.471933 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-ce291b26-7ed3-4c9f-95bf-fab2bc22eb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:35.548433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 572/1132] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:35.574874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fabaa4-ee1c-4d78-aad2-ac129e35969f None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:35.687441 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fabaa4-ee1c-4d78-aad2-ac129e35969f None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:35.725220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e71577-7d1b-4468-b77a-9ec552eb253f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:35.725940 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e71577-7d1b-4468-b77a-9ec552eb253f None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:35.726217 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e71577-7d1b-4468-b77a-9ec552eb253f None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:35.726487 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e71577-7d1b-4468-b77a-9ec552eb253f None admin] PATH_INFO: `/v3/domains/default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:35.727745 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e71577-7d1b-4468-b77a-9ec552eb253f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:35.729823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e71577-7d1b-4468-b77a-9ec552eb253f None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:35.731579 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 561/1133] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:27:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:27:35.737935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce291b26-7ed3-4c9f-95bf-fab2bc22eb40 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:35.846402 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce291b26-7ed3-4c9f-95bf-fab2bc22eb40 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:35.880159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:35.885872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79013e62-fa3b-4776-8e4d-48c540900e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:35.886543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79013e62-fa3b-4776-8e4d-48c540900e6b None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:35.886829 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79013e62-fa3b-4776-8e4d-48c540900e6b None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:35.887106 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79013e62-fa3b-4776-8e4d-48c540900e6b None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:35.887840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79013e62-fa3b-4776-8e4d-48c540900e6b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:35.889316 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79013e62-fa3b-4776-8e4d-48c540900e6b None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:35.918985 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 573/1134] 10.222.0.71 () {60 vars in 1141 bytes} [Thu Jul 21 20:27:35 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:27:38.021399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08e9e1-c170-4e9c-85c3-f9b0f7d43df8 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:38.021953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08e9e1-c170-4e9c-85c3-f9b0f7d43df8 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:38.021953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08e9e1-c170-4e9c-85c3-f9b0f7d43df8 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:38.023314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 562/1135] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:38.029214 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7e425-8cf3-43b4-8f63-93cf904999ef None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:38.029629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7e425-8cf3-43b4-8f63-93cf904999ef None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:38.029762 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7e425-8cf3-43b4-8f63-93cf904999ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:38.046677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-70a7e425-8cf3-43b4-8f63-93cf904999ef None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:38.047862 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-70a7e425-8cf3-43b4-8f63-93cf904999ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:38.112776 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 574/1136] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:38.136987 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08e9e1-c170-4e9c-85c3-f9b0f7d43df8 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:38.146117 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:38.238301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08e9e1-c170-4e9c-85c3-f9b0f7d43df8 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:38.270571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d7160-7c51-4664-b3bd-ceffc98a434a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:38.271353 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d7160-7c51-4664-b3bd-ceffc98a434a None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:38.271686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d7160-7c51-4664-b3bd-ceffc98a434a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:38.271910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d7160-7c51-4664-b3bd-ceffc98a434a None admin] PATH_INFO: `/v3/domains/default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:38.273104 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d7160-7c51-4664-b3bd-ceffc98a434a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:38.275046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d7160-7c51-4664-b3bd-ceffc98a434a None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:38.276672 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 563/1137] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:27:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:38.282526 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7e425-8cf3-43b4-8f63-93cf904999ef None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:38.378777 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7e425-8cf3-43b4-8f63-93cf904999ef None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:38.392633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:38.413446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa464077-5438-438e-8809-d88e167d3f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:38.414104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa464077-5438-438e-8809-d88e167d3f2e None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:38.414384 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa464077-5438-438e-8809-d88e167d3f2e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:38.414657 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa464077-5438-438e-8809-d88e167d3f2e None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:38.415416 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa464077-5438-438e-8809-d88e167d3f2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:38.416844 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa464077-5438-438e-8809-d88e167d3f2e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:38.435571 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 575/1138] 10.222.0.71 () {60 vars in 1132 bytes} [Thu Jul 21 20:27:38 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:27:40.537140 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd42ea-7ac9-4530-b19c-742e54dcd334 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:40.537667 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd42ea-7ac9-4530-b19c-742e54dcd334 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:40.537667 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd42ea-7ac9-4530-b19c-742e54dcd334 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:40.539050 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 564/1139] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:40.544801 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75a8f8-4cbc-4088-86f2-b002887bb950 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:40.545245 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75a8f8-4cbc-4088-86f2-b002887bb950 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:40.545382 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75a8f8-4cbc-4088-86f2-b002887bb950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:40.563349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-3b75a8f8-4cbc-4088-86f2-b002887bb950 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:40.564545 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3b75a8f8-4cbc-4088-86f2-b002887bb950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:40.630833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 576/1140] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:40.653502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd42ea-7ac9-4530-b19c-742e54dcd334 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:40.729048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:40.755049 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd42ea-7ac9-4530-b19c-742e54dcd334 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:40.787445 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27088a5-77d3-46fd-b006-a84fa9198b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:40.788226 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27088a5-77d3-46fd-b006-a84fa9198b7c None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:40.788511 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27088a5-77d3-46fd-b006-a84fa9198b7c None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:40.788787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27088a5-77d3-46fd-b006-a84fa9198b7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:40.799285 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27088a5-77d3-46fd-b006-a84fa9198b7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:27:40.807447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-a27088a5-77d3-46fd-b006-a84fa9198b7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:27:40.807447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:40.807447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:40.807447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:40.807447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:40.807447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:40.807447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:40.807447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:40.807447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:40.807447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:40.807447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:40.807447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:40.807447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:27:40.807447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:27:40.807447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:27:40.807447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:27:40.808554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:27:40.808554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:27:40.808554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:27:40.808554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:27:40.808554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:40.808554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:40.808554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:40.808554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:40.808554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:40.808554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:40.808554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:40.808554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:40.808554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:40.808554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:40.808554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:40.808554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:40.809642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:40.809642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:40.809642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:40.809642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:40.809642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:27:40.809642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:27:40.809642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:27:40.809642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:27:40.809642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:27:40.809642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:27:40.809642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:27:40.809642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:40.809642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 565/1141] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:27:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:40.814676 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75a8f8-4cbc-4088-86f2-b002887bb950 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:40.909257 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:40.912885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75a8f8-4cbc-4088-86f2-b002887bb950 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:40.947401 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447fd58-bb04-4a6e-a333-8c15e28b2147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:40.948072 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447fd58-bb04-4a6e-a333-8c15e28b2147 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:40.948353 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447fd58-bb04-4a6e-a333-8c15e28b2147 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:40.948627 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447fd58-bb04-4a6e-a333-8c15e28b2147 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:40.949407 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447fd58-bb04-4a6e-a333-8c15e28b2147 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:40.949718 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447fd58-bb04-4a6e-a333-8c15e28b2147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:40.951097 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447fd58-bb04-4a6e-a333-8c15e28b2147 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:40.959663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 577/1142] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:27:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:40.966149 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27088a5-77d3-46fd-b006-a84fa9198b7c None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:41.064354 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27088a5-77d3-46fd-b006-a84fa9198b7c None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:41.099186 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78611d62-67c8-4412-9716-7697bcf9b844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:41.100059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78611d62-67c8-4412-9716-7697bcf9b844 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:41.100339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78611d62-67c8-4412-9716-7697bcf9b844 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:41.100615 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78611d62-67c8-4412-9716-7697bcf9b844 None admin] PATH_INFO: `/v3/users/d500a7750c5648519eab3de0f958677d` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:41.102711 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:41.116302 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78611d62-67c8-4412-9716-7697bcf9b844 None admin] RBAC: Authorizing `identity:get_user(user_id=d500a7750c5648519eab3de0f958677d)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:41.117762 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78611d62-67c8-4412-9716-7697bcf9b844 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:41.119537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 566/1143] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:27:40 2022] GET /identity/v3/users/d500a7750c5648519eab3de0f958677d => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:41.125629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447fd58-bb04-4a6e-a333-8c15e28b2147 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:41.229518 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447fd58-bb04-4a6e-a333-8c15e28b2147 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:41.263736 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1bf99-6134-43bb-9735-e20173200d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:41.264380 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1bf99-6134-43bb-9735-e20173200d94 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:41.264665 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1bf99-6134-43bb-9735-e20173200d94 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:41.264944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1bf99-6134-43bb-9735-e20173200d94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:41.296215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:41.335775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1bf99-6134-43bb-9735-e20173200d94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:41.337735 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1bf99-6134-43bb-9735-e20173200d94 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:41.438569 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 578/1144] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:27:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:27:41.445261 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78611d62-67c8-4412-9716-7697bcf9b844 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:41.551988 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78611d62-67c8-4412-9716-7697bcf9b844 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:41.588385 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aabdc6-ce1e-4a60-bf4d-d7e78b432415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:41.589180 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aabdc6-ce1e-4a60-bf4d-d7e78b432415 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:41.589566 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aabdc6-ce1e-4a60-bf4d-d7e78b432415 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:41.589838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aabdc6-ce1e-4a60-bf4d-d7e78b432415 None admin] PATH_INFO: `/v3/projects/172feb039c52477c88c70b356e65e4d1` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:41.591722 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aabdc6-ce1e-4a60-bf4d-d7e78b432415 None admin] RBAC: Authorizing `identity:get_project(project_id=172feb039c52477c88c70b356e65e4d1)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:41.594389 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aabdc6-ce1e-4a60-bf4d-d7e78b432415 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:41.597864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 567/1145] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:27:41 2022] GET /identity/v3/projects/172feb039c52477c88c70b356e65e4d1 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:41.604405 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1bf99-6134-43bb-9735-e20173200d94 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:41.672349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:41.718362 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1bf99-6134-43bb-9735-e20173200d94 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:41.754392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f371d-8b25-4a3a-bbac-fd471947d617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:41.755046 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f371d-8b25-4a3a-bbac-fd471947d617 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:41.755329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f371d-8b25-4a3a-bbac-fd471947d617 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:41.755609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f371d-8b25-4a3a-bbac-fd471947d617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:41.756440 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f371d-8b25-4a3a-bbac-fd471947d617 None admin] RBAC: Adding query filter params (role.id=19f5e72c0be44a1197c7288de82e79e8, scope.project.id=172feb039c52477c88c70b356e65e4d1, user.id=d500a7750c5648519eab3de0f958677d) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:41.756754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f371d-8b25-4a3a-bbac-fd471947d617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:41.758230 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f371d-8b25-4a3a-bbac-fd471947d617 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:41.764445 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 579/1146] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:27:41 2022] GET /identity/v3/role_assignments?user.id=d500a7750c5648519eab3de0f958677d&scope.project.id=172feb039c52477c88c70b356e65e4d1&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:43.834894 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991fbc2-ca67-41a5-b565-4901bd3e51e6 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:43.835399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991fbc2-ca67-41a5-b565-4901bd3e51e6 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:43.835399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991fbc2-ca67-41a5-b565-4901bd3e51e6 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:43.836810 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 568/1147] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:27:43.843485 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07e99e-fbb1-4c3a-8f58-9d7ae51a1279 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:43.844022 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07e99e-fbb1-4c3a-8f58-9d7ae51a1279 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:43.844022 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07e99e-fbb1-4c3a-8f58-9d7ae51a1279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:43.861892 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-6f07e99e-fbb1-4c3a-8f58-9d7ae51a1279 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:43.863112 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-6f07e99e-fbb1-4c3a-8f58-9d7ae51a1279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:43.936865 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 580/1148] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:43.960655 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991fbc2-ca67-41a5-b565-4901bd3e51e6 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:44.024888 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:44.066254 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991fbc2-ca67-41a5-b565-4901bd3e51e6 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:44.099698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb7764-4858-437f-98d3-2f9106b981f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:44.100397 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb7764-4858-437f-98d3-2f9106b981f8 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:44.100673 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb7764-4858-437f-98d3-2f9106b981f8 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:44.100942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb7764-4858-437f-98d3-2f9106b981f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:44.111985 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb7764-4858-437f-98d3-2f9106b981f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:27:44.119660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-94cb7764-4858-437f-98d3-2f9106b981f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:27:44.119660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:44.119660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:44.119660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:44.119660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:44.119660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:44.119660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:44.119660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:44.119660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:44.119660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:44.119660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:44.119660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:44.119660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:27:44.119660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:27:44.119660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:27:44.119660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:27:44.120791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:27:44.120791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:27:44.120791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:27:44.120791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:27:44.120791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:44.120791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:44.120791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:44.120791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:44.120791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:44.120791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:44.120791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:44.120791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:44.120791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:44.120791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:44.120791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:44.120791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:44.121931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:44.121931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:44.121931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:44.121931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:44.121931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:27:44.121931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:27:44.121931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:27:44.121931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:27:44.121931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:27:44.121931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:27:44.121931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:27:44.121931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:44.121931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 569/1149] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:27:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:44.127186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07e99e-fbb1-4c3a-8f58-9d7ae51a1279 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:44.184232 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:44.228865 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07e99e-fbb1-4c3a-8f58-9d7ae51a1279 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:44.262348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7a5f1-34bb-4b14-b48b-8b18836aaead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:44.263004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7a5f1-34bb-4b14-b48b-8b18836aaead None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:44.263288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7a5f1-34bb-4b14-b48b-8b18836aaead None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:44.263565 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7a5f1-34bb-4b14-b48b-8b18836aaead None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:44.264308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7a5f1-34bb-4b14-b48b-8b18836aaead None admin] RBAC: Adding query filter params (name=admin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:44.264616 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7a5f1-34bb-4b14-b48b-8b18836aaead None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:44.266024 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7a5f1-34bb-4b14-b48b-8b18836aaead None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:44.274000 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 581/1150] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:27:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:44.280516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb7764-4858-437f-98d3-2f9106b981f8 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:44.383438 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb7764-4858-437f-98d3-2f9106b981f8 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:44.403880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:44.418110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eaabe4-7ae1-4873-adfd-835e7d1b0259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:44.419010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eaabe4-7ae1-4873-adfd-835e7d1b0259 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:44.419277 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eaabe4-7ae1-4873-adfd-835e7d1b0259 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:44.419550 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eaabe4-7ae1-4873-adfd-835e7d1b0259 None admin] PATH_INFO: `/v3/users/d500a7750c5648519eab3de0f958677d` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:44.420944 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eaabe4-7ae1-4873-adfd-835e7d1b0259 None admin] RBAC: Authorizing `identity:get_user(user_id=d500a7750c5648519eab3de0f958677d)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:44.422413 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eaabe4-7ae1-4873-adfd-835e7d1b0259 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:44.424065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 570/1151] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:27:44 2022] GET /identity/v3/users/d500a7750c5648519eab3de0f958677d => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:44.430014 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7a5f1-34bb-4b14-b48b-8b18836aaead None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:44.529889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7a5f1-34bb-4b14-b48b-8b18836aaead None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:44.554106 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:44.570694 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e6f5c-8b90-448f-bfbd-b18351c5accd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:44.572055 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e6f5c-8b90-448f-bfbd-b18351c5accd None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:44.572333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e6f5c-8b90-448f-bfbd-b18351c5accd None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:44.572606 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e6f5c-8b90-448f-bfbd-b18351c5accd None admin] PATH_INFO: `/v3/projects/172feb039c52477c88c70b356e65e4d1` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:44.573795 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e6f5c-8b90-448f-bfbd-b18351c5accd None admin] RBAC: Authorizing `identity:get_project(project_id=172feb039c52477c88c70b356e65e4d1)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:44.575226 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e6f5c-8b90-448f-bfbd-b18351c5accd None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:44.576906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 582/1152] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:27:44 2022] GET /identity/v3/projects/172feb039c52477c88c70b356e65e4d1 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:44.582902 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eaabe4-7ae1-4873-adfd-835e7d1b0259 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:44.679509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eaabe4-7ae1-4873-adfd-835e7d1b0259 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:44.712358 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2330ddd-c0f3-44b5-9cfe-603e7da070c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:44.713245 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2330ddd-c0f3-44b5-9cfe-603e7da070c9 None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:44.713523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2330ddd-c0f3-44b5-9cfe-603e7da070c9 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:44.713797 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2330ddd-c0f3-44b5-9cfe-603e7da070c9 None admin] PATH_INFO: `/v3/projects/172feb039c52477c88c70b356e65e4d1/users/d500a7750c5648519eab3de0f958677d/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:44.715702 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2330ddd-c0f3-44b5-9cfe-603e7da070c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=172feb039c52477c88c70b356e65e4d1, user_id=d500a7750c5648519eab3de0f958677d, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:44.717145 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2330ddd-c0f3-44b5-9cfe-603e7da070c9 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:44.730115 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 571/1153] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:27:44 2022] PUT /identity/v3/projects/172feb039c52477c88c70b356e65e4d1/users/d500a7750c5648519eab3de0f958677d/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:27:46.748962 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97c7e1-66dd-4ee6-b8ab-fef28fe7ef30 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:46.749561 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97c7e1-66dd-4ee6-b8ab-fef28fe7ef30 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:46.749561 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97c7e1-66dd-4ee6-b8ab-fef28fe7ef30 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:46.750808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 583/1154] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:46.755943 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c701d4a-cb25-4e10-b519-73538cc555e3 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:46.756393 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c701d4a-cb25-4e10-b519-73538cc555e3 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:46.756497 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c701d4a-cb25-4e10-b519-73538cc555e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:46.773435 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-0c701d4a-cb25-4e10-b519-73538cc555e3 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:46.775213 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-0c701d4a-cb25-4e10-b519-73538cc555e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:46.816293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:46.840268 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 572/1155] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:46.863795 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97c7e1-66dd-4ee6-b8ab-fef28fe7ef30 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:46.963482 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97c7e1-66dd-4ee6-b8ab-fef28fe7ef30 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:46.996596 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8239ca-bc3b-4203-8696-a4f75c891aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:46.997396 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8239ca-bc3b-4203-8696-a4f75c891aea None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:46.997677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8239ca-bc3b-4203-8696-a4f75c891aea None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:46.997947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8239ca-bc3b-4203-8696-a4f75c891aea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:47.007864 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8239ca-bc3b-4203-8696-a4f75c891aea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:27:47.015880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-0d8239ca-bc3b-4203-8696-a4f75c891aea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:27:47.015880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:47.015880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:47.015880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:47.015880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:47.015880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:47.015880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:47.015880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:47.015880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:47.015880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:47.015880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:47.015880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:47.015880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:27:47.015880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:27:47.015880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:27:47.015880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:27:47.017096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:27:47.017096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:27:47.017096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:27:47.017096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:27:47.017096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:47.017096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:47.017096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:47.017096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:47.017096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:47.017096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:47.017096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:47.017096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:47.017096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:47.017096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:47.017096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:47.017096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:47.018139 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:47.018139 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:47.018139 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:47.018139 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:47.018139 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:27:47.018139 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:27:47.018139 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:27:47.018139 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:27:47.018139 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:27:47.018139 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:27:47.018139 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:27:47.018139 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:27:47.018139 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 584/1156] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:27:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:47.023838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c701d4a-cb25-4e10-b519-73538cc555e3 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:47.120556 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c701d4a-cb25-4e10-b519-73538cc555e3 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:47.155400 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7443a77-84f9-4117-b31e-2a5a0c00a752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:47.156074 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7443a77-84f9-4117-b31e-2a5a0c00a752 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:47.156360 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7443a77-84f9-4117-b31e-2a5a0c00a752 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:47.156640 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7443a77-84f9-4117-b31e-2a5a0c00a752 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:47.157424 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7443a77-84f9-4117-b31e-2a5a0c00a752 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:47.157738 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7443a77-84f9-4117-b31e-2a5a0c00a752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:47.159107 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7443a77-84f9-4117-b31e-2a5a0c00a752 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:47.167449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 573/1157] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:27:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:47.173326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8239ca-bc3b-4203-8696-a4f75c891aea None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:47.220565 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:47.272725 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8239ca-bc3b-4203-8696-a4f75c891aea None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:47.304914 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f11c6d-d60c-439b-b0c3-0057b099fa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:47.306414 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f11c6d-d60c-439b-b0c3-0057b099fa37 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:47.306695 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f11c6d-d60c-439b-b0c3-0057b099fa37 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:47.306967 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f11c6d-d60c-439b-b0c3-0057b099fa37 None admin] PATH_INFO: `/v3/users/d500a7750c5648519eab3de0f958677d` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:47.308140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f11c6d-d60c-439b-b0c3-0057b099fa37 None admin] RBAC: Authorizing `identity:get_user(user_id=d500a7750c5648519eab3de0f958677d)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:47.309610 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f11c6d-d60c-439b-b0c3-0057b099fa37 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:47.311232 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 585/1158] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:27:47 2022] GET /identity/v3/users/d500a7750c5648519eab3de0f958677d => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:47.317048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7443a77-84f9-4117-b31e-2a5a0c00a752 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:47.358570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:47.416311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7443a77-84f9-4117-b31e-2a5a0c00a752 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:47.449370 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6ce90-b7f4-472b-8024-0a14cc568e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:47.450019 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6ce90-b7f4-472b-8024-0a14cc568e99 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:47.450299 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6ce90-b7f4-472b-8024-0a14cc568e99 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:47.450571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6ce90-b7f4-472b-8024-0a14cc568e99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:47.516888 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6ce90-b7f4-472b-8024-0a14cc568e99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:47.518365 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6ce90-b7f4-472b-8024-0a14cc568e99 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:47.556886 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:47.614263 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 574/1159] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:27:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:27:47.620401 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f11c6d-d60c-439b-b0c3-0057b099fa37 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:47.717643 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f11c6d-d60c-439b-b0c3-0057b099fa37 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:47.730737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:47.750590 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4a761-e40b-4ba7-877e-516aefa46c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:47.751425 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4a761-e40b-4ba7-877e-516aefa46c35 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:47.751701 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4a761-e40b-4ba7-877e-516aefa46c35 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:47.751972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4a761-e40b-4ba7-877e-516aefa46c35 None admin] PATH_INFO: `/v3/projects/172feb039c52477c88c70b356e65e4d1` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:47.753141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4a761-e40b-4ba7-877e-516aefa46c35 None admin] RBAC: Authorizing `identity:get_project(project_id=172feb039c52477c88c70b356e65e4d1)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:47.754569 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4a761-e40b-4ba7-877e-516aefa46c35 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:47.756238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 586/1160] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:27:47 2022] GET /identity/v3/projects/172feb039c52477c88c70b356e65e4d1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:47.763057 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6ce90-b7f4-472b-8024-0a14cc568e99 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:47.858509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6ce90-b7f4-472b-8024-0a14cc568e99 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:47.891838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204adf2-dc33-4e97-acec-3abb2b86b90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:47.892507 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204adf2-dc33-4e97-acec-3abb2b86b90d None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:47.892764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204adf2-dc33-4e97-acec-3abb2b86b90d None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:47.893039 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204adf2-dc33-4e97-acec-3abb2b86b90d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:47.893897 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1204adf2-dc33-4e97-acec-3abb2b86b90d None admin] RBAC: Adding query filter params (role.id=19f5e72c0be44a1197c7288de82e79e8, scope.project.id=172feb039c52477c88c70b356e65e4d1, user.id=d500a7750c5648519eab3de0f958677d) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:47.894211 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1204adf2-dc33-4e97-acec-3abb2b86b90d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:47.895635 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1204adf2-dc33-4e97-acec-3abb2b86b90d None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:47.901405 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 575/1161] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:27:47 2022] GET /identity/v3/role_assignments?user.id=d500a7750c5648519eab3de0f958677d&scope.project.id=172feb039c52477c88c70b356e65e4d1&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:49.976948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b19d48-c975-48a1-82e5-4a8f967b4bd6 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:49.977513 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b19d48-c975-48a1-82e5-4a8f967b4bd6 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:49.977620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b19d48-c975-48a1-82e5-4a8f967b4bd6 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:49.978938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 587/1162] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:49.984107 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0f3b6-ee31-400e-a9e0-9be2116e1111 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:49.984494 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0f3b6-ee31-400e-a9e0-9be2116e1111 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:49.984654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0f3b6-ee31-400e-a9e0-9be2116e1111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:49.990569 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:50.002116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-2fb0f3b6-ee31-400e-a9e0-9be2116e1111 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:50.003293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-2fb0f3b6-ee31-400e-a9e0-9be2116e1111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:50.068292 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 576/1163] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:27:50.091362 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b19d48-c975-48a1-82e5-4a8f967b4bd6 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:50.190826 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b19d48-c975-48a1-82e5-4a8f967b4bd6 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:50.224393 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6199f3b-661d-42f2-9d9f-9d15a35e4dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:50.225033 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6199f3b-661d-42f2-9d9f-9d15a35e4dc0 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:50.225354 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6199f3b-661d-42f2-9d9f-9d15a35e4dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:50.225629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6199f3b-661d-42f2-9d9f-9d15a35e4dc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:50.273324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:50.292254 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6199f3b-661d-42f2-9d9f-9d15a35e4dc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:50.293754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6199f3b-661d-42f2-9d9f-9d15a35e4dc0 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:50.390201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 588/1164] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:27:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:27:50.396395 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0f3b6-ee31-400e-a9e0-9be2116e1111 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:50.494428 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0f3b6-ee31-400e-a9e0-9be2116e1111 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:50.527303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf784e-dc82-407a-91ce-2b276db30893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:50.528026 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf784e-dc82-407a-91ce-2b276db30893 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:50.528311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf784e-dc82-407a-91ce-2b276db30893 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:50.528586 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf784e-dc82-407a-91ce-2b276db30893 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:50.530475 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:50.536421 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf784e-dc82-407a-91ce-2b276db30893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:50.538102 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf784e-dc82-407a-91ce-2b276db30893 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:50.552596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-47bf784e-dc82-407a-91ce-2b276db30893 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 20:27:50.552596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:50.552596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 21 20:27:50.552596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 21 20:27:50.552596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:27:50.552596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:27:50.552596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:27:50.552596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:27:50.552596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 21 20:27:50.552596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:50.552596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:27:50.552596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:50.552596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:27:50.552596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:27:50.552596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:27:50.552596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:27:50.552596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:27:50.553921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:27:50.553921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:27:50.553921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:27:50.553921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:27:50.553921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:27:50.553921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:27:50.553921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:27:50.553921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:27:50.553921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:27:50.553921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:27:50.553921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:27:50.553921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:27:50.553921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:27:50.553921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:27:50.553921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:27:50.553921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:27:50.555096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:27:50.555096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:27:50.555096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:27:50.555096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:27:50.555096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:27:50.555096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:27:50.555096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:27:50.555096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:27:50.555096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:27:50.555096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:27:50.555096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 21 20:27:50.555096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:27:50.555096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 20:27:50.555096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:27:50.555096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 577/1165] 10.222.0.71 () {56 vars in 1117 bytes} [Thu Jul 21 20:27:50 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:27:50.560428 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6199f3b-661d-42f2-9d9f-9d15a35e4dc0 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:50.642003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:50.660304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6199f3b-661d-42f2-9d9f-9d15a35e4dc0 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:50.692957 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9aaa31-39f1-4132-a03b-b59ddd76b77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:50.693623 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9aaa31-39f1-4132-a03b-b59ddd76b77f None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:50.693903 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9aaa31-39f1-4132-a03b-b59ddd76b77f None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:50.694176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9aaa31-39f1-4132-a03b-b59ddd76b77f None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:50.694895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9aaa31-39f1-4132-a03b-b59ddd76b77f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:27:50.695200 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9aaa31-39f1-4132-a03b-b59ddd76b77f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:50.697195 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9aaa31-39f1-4132-a03b-b59ddd76b77f None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:50.704162 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 589/1166] 10.222.0.71 () {56 vars in 1115 bytes} [Thu Jul 21 20:27:50 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:52.715259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb672795-4c88-41a7-a162-dbfc4c06399e None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:52.715779 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb672795-4c88-41a7-a162-dbfc4c06399e None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:52.715779 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb672795-4c88-41a7-a162-dbfc4c06399e None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:52.717105 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 578/1167] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:52.722496 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d46b8-fe73-4b13-9ece-c464c292ff2f None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:52.722907 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d46b8-fe73-4b13-9ece-c464c292ff2f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:52.723055 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d46b8-fe73-4b13-9ece-c464c292ff2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:52.739521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-5f2d46b8-fe73-4b13-9ece-c464c292ff2f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:52.740712 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-5f2d46b8-fe73-4b13-9ece-c464c292ff2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:52.805703 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 590/1168] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:52.828569 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb672795-4c88-41a7-a162-dbfc4c06399e None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:52.927314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb672795-4c88-41a7-a162-dbfc4c06399e None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:52.960319 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1fadf-ac12-472b-898d-ee90e49f43e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:52.960981 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1fadf-ac12-472b-898d-ee90e49f43e2 None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:52.961292 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1fadf-ac12-472b-898d-ee90e49f43e2 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:52.962108 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1fadf-ac12-472b-898d-ee90e49f43e2 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:52.962859 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1fadf-ac12-472b-898d-ee90e49f43e2 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:52.964227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1fadf-ac12-472b-898d-ee90e49f43e2 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:52.979804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 579/1169] 10.222.0.71 () {60 vars in 1138 bytes} [Thu Jul 21 20:27:52 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:27:55.057357 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788c67c-4354-4938-b11b-208d00ad82c0 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:55.057910 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788c67c-4354-4938-b11b-208d00ad82c0 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:55.057910 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788c67c-4354-4938-b11b-208d00ad82c0 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:55.059278 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 591/1170] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:55.064426 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135de163-37df-4f64-9ad5-7f6569bebeb2 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:55.064835 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135de163-37df-4f64-9ad5-7f6569bebeb2 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:55.064993 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135de163-37df-4f64-9ad5-7f6569bebeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:55.081939 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-135de163-37df-4f64-9ad5-7f6569bebeb2 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:55.083119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-135de163-37df-4f64-9ad5-7f6569bebeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:55.131298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:55.150731 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 580/1171] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:55.174047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788c67c-4354-4938-b11b-208d00ad82c0 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:55.251251 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:55.274792 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788c67c-4354-4938-b11b-208d00ad82c0 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:55.307688 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653730c3-b147-4c39-9152-4dc29033c5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:55.308467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653730c3-b147-4c39-9152-4dc29033c5eb None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:55.308747 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653730c3-b147-4c39-9152-4dc29033c5eb None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:55.309018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653730c3-b147-4c39-9152-4dc29033c5eb None admin] PATH_INFO: `/v3/domains/5ad53e5cf3eb4e189566b5c1ddb298e4` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:55.319069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653730c3-b147-4c39-9152-4dc29033c5eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=5ad53e5cf3eb4e189566b5c1ddb298e4)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:55.320509 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653730c3-b147-4c39-9152-4dc29033c5eb None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:55.322186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 592/1172] 10.222.0.71 () {56 vars in 1183 bytes} [Thu Jul 21 20:27:55 2022] GET /identity/v3/domains/5ad53e5cf3eb4e189566b5c1ddb298e4 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:55.328101 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135de163-37df-4f64-9ad5-7f6569bebeb2 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:55.424899 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135de163-37df-4f64-9ad5-7f6569bebeb2 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:55.458279 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb38e5-1ab5-4704-b591-324054b0b926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:55.458939 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb38e5-1ab5-4704-b591-324054b0b926 None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:55.459224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb38e5-1ab5-4704-b591-324054b0b926 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:55.459502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb38e5-1ab5-4704-b591-324054b0b926 None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:55.460213 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb38e5-1ab5-4704-b591-324054b0b926 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:55.462226 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb38e5-1ab5-4704-b591-324054b0b926 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:55.489061 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 581/1173] 10.222.0.71 () {60 vars in 1141 bytes} [Thu Jul 21 20:27:55 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:27:57.563972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b3cf0-5869-4f08-b3b5-da5dcd5e6274 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:57.564512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b3cf0-5869-4f08-b3b5-da5dcd5e6274 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:57.564512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b3cf0-5869-4f08-b3b5-da5dcd5e6274 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:57.565920 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 593/1174] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:27:57.571298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8c553-3e72-479a-aab3-da71105f10bf None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:57.571742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8c553-3e72-479a-aab3-da71105f10bf None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:57.571847 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8c553-3e72-479a-aab3-da71105f10bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:57.588721 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-77d8c553-3e72-479a-aab3-da71105f10bf None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:27:57.589946 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-77d8c553-3e72-479a-aab3-da71105f10bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:27:57.618631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:27:57.656709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 582/1175] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:27:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:27:57.682061 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b3cf0-5869-4f08-b3b5-da5dcd5e6274 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:57.780238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b3cf0-5869-4f08-b3b5-da5dcd5e6274 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:57.812637 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9948ee9-bc9a-4de8-b64f-2ecfbc60ab56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:57.813498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9948ee9-bc9a-4de8-b64f-2ecfbc60ab56 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:57.813773 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9948ee9-bc9a-4de8-b64f-2ecfbc60ab56 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:57.814039 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9948ee9-bc9a-4de8-b64f-2ecfbc60ab56 None admin] PATH_INFO: `/v3/domains/5ad53e5cf3eb4e189566b5c1ddb298e4` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:57.815190 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9948ee9-bc9a-4de8-b64f-2ecfbc60ab56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5ad53e5cf3eb4e189566b5c1ddb298e4)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:57.816610 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9948ee9-bc9a-4de8-b64f-2ecfbc60ab56 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:57.818230 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 594/1176] 10.222.0.71 () {56 vars in 1183 bytes} [Thu Jul 21 20:27:57 2022] GET /identity/v3/domains/5ad53e5cf3eb4e189566b5c1ddb298e4 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:27:57.824646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8c553-3e72-479a-aab3-da71105f10bf None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:27:57.920770 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8c553-3e72-479a-aab3-da71105f10bf None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:27:57.955808 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048a9da-702f-41f7-89ba-a2a69fbe723c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:27:57.956481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048a9da-702f-41f7-89ba-a2a69fbe723c None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:27:57.956762 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048a9da-702f-41f7-89ba-a2a69fbe723c None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:27:57.957035 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048a9da-702f-41f7-89ba-a2a69fbe723c None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:27:57.957789 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048a9da-702f-41f7-89ba-a2a69fbe723c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:27:57.959223 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048a9da-702f-41f7-89ba-a2a69fbe723c None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:27:57.979489 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 583/1177] 10.222.0.71 () {60 vars in 1132 bytes} [Thu Jul 21 20:27:57 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:28:00.060619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f139f8-be52-475d-abc4-b8f329774f74 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:00.061202 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f139f8-be52-475d-abc4-b8f329774f74 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:00.061318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f139f8-be52-475d-abc4-b8f329774f74 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:00.062669 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 595/1178] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:28:00.068242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35633129-3c34-437f-8775-daf1dc81e1ec None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:00.068772 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35633129-3c34-437f-8775-daf1dc81e1ec None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:00.068875 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35633129-3c34-437f-8775-daf1dc81e1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:00.085677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-35633129-3c34-437f-8775-daf1dc81e1ec None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:28:00.087491 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-35633129-3c34-437f-8775-daf1dc81e1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:28:00.103847 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:28:00.156192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 584/1179] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:28:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:28:00.180291 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f139f8-be52-475d-abc4-b8f329774f74 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:28:00.184147 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:28:00.287200 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f139f8-be52-475d-abc4-b8f329774f74 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:28:00.319744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d132074-7b93-4ffb-aa36-0abc5b422009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:28:00.320951 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d132074-7b93-4ffb-aa36-0abc5b422009 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:00.321279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d132074-7b93-4ffb-aa36-0abc5b422009 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:00.321551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d132074-7b93-4ffb-aa36-0abc5b422009 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:00.331472 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d132074-7b93-4ffb-aa36-0abc5b422009 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:28:00.339605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-0d132074-7b93-4ffb-aa36-0abc5b422009 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:28:00.339605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:28:00.339605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:28:00.339605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:28:00.339605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:28:00.339605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:28:00.339605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:28:00.339605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:28:00.339605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:28:00.339605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:28:00.339605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:28:00.339605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:28:00.339605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:28:00.339605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:28:00.339605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:28:00.339605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:28:00.340771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:28:00.340771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:28:00.340771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:28:00.340771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:28:00.340771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:28:00.340771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:28:00.340771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:28:00.340771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:28:00.340771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:28:00.340771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:28:00.340771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:28:00.340771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:28:00.340771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:28:00.340771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:28:00.340771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:28:00.340771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:28:00.342124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:28:00.342124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:28:00.342124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:28:00.342124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:28:00.342124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:28:00.342124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:28:00.342124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:28:00.342124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:28:00.342124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:28:00.342124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:28:00.342124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:28:00.342124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:28:00.342124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 596/1180] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:28:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:28:00.347374 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35633129-3c34-437f-8775-daf1dc81e1ec None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:28:00.441148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35633129-3c34-437f-8775-daf1dc81e1ec None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:28:00.475155 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d3dde-e50d-4267-8680-9cb759955082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:28:00.475819 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d3dde-e50d-4267-8680-9cb759955082 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:00.476097 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d3dde-e50d-4267-8680-9cb759955082 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:00.476368 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d3dde-e50d-4267-8680-9cb759955082 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:00.477219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d3dde-e50d-4267-8680-9cb759955082 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:28:00.477523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d3dde-e50d-4267-8680-9cb759955082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:28:00.478879 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d3dde-e50d-4267-8680-9cb759955082 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:28:00.487473 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 585/1181] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:28:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:28:00.493834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d132074-7b93-4ffb-aa36-0abc5b422009 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:28:00.553860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:28:00.592413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d132074-7b93-4ffb-aa36-0abc5b422009 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:28:00.625569 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dc100-53f0-4730-b0b8-f03e4ba67193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:28:00.626428 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dc100-53f0-4730-b0b8-f03e4ba67193 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:00.626710 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dc100-53f0-4730-b0b8-f03e4ba67193 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:00.626987 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dc100-53f0-4730-b0b8-f03e4ba67193 None admin] PATH_INFO: `/v3/users/cac7cb96a9ef4e30bb6c3c309919ac00` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:00.640294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dc100-53f0-4730-b0b8-f03e4ba67193 None admin] RBAC: Authorizing `identity:get_user(user_id=cac7cb96a9ef4e30bb6c3c309919ac00)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:28:00.641774 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dc100-53f0-4730-b0b8-f03e4ba67193 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:28:00.643423 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 597/1182] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:28:00 2022] GET /identity/v3/users/cac7cb96a9ef4e30bb6c3c309919ac00 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:28:00.649356 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d3dde-e50d-4267-8680-9cb759955082 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:28:00.665195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:28:00.747046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d3dde-e50d-4267-8680-9cb759955082 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:28:00.779735 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd6a23-893b-475c-a195-27b515b94767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:28:00.780375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd6a23-893b-475c-a195-27b515b94767 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:00.780657 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd6a23-893b-475c-a195-27b515b94767 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:00.780929 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd6a23-893b-475c-a195-27b515b94767 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:00.845274 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd6a23-893b-475c-a195-27b515b94767 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:28:00.846900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd6a23-893b-475c-a195-27b515b94767 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:28:00.861717 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:28:00.941996 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 586/1183] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:28:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:28:00.948209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dc100-53f0-4730-b0b8-f03e4ba67193 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:28:01.044664 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dc100-53f0-4730-b0b8-f03e4ba67193 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:28:01.066876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:28:01.078704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6363e3-dd2b-4ada-84a7-4f1bd3dd360e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:28:01.079551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6363e3-dd2b-4ada-84a7-4f1bd3dd360e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:01.079834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6363e3-dd2b-4ada-84a7-4f1bd3dd360e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:01.080110 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6363e3-dd2b-4ada-84a7-4f1bd3dd360e None admin] PATH_INFO: `/v3/projects/194254728c914ec7b65ea0f830811faf` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:01.081317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6363e3-dd2b-4ada-84a7-4f1bd3dd360e None admin] RBAC: Authorizing `identity:get_project(project_id=194254728c914ec7b65ea0f830811faf)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:28:01.082752 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6363e3-dd2b-4ada-84a7-4f1bd3dd360e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:28:01.084433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 598/1184] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:28:00 2022] GET /identity/v3/projects/194254728c914ec7b65ea0f830811faf => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:28:01.090597 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd6a23-893b-475c-a195-27b515b94767 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:28:01.188651 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd6a23-893b-475c-a195-27b515b94767 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:28:01.202327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:28:01.223191 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed4972-c38f-4890-9724-d1970645af34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:28:01.223843 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed4972-c38f-4890-9724-d1970645af34 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:01.224125 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed4972-c38f-4890-9724-d1970645af34 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:01.224402 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed4972-c38f-4890-9724-d1970645af34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:01.225262 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed4972-c38f-4890-9724-d1970645af34 None admin] RBAC: Adding query filter params (role.id=19f5e72c0be44a1197c7288de82e79e8, scope.project.id=194254728c914ec7b65ea0f830811faf, user.id=cac7cb96a9ef4e30bb6c3c309919ac00) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:28:01.225576 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed4972-c38f-4890-9724-d1970645af34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:28:01.227558 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed4972-c38f-4890-9724-d1970645af34 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:28:01.233004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 587/1185] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:28:01 2022] GET /identity/v3/role_assignments?user.id=cac7cb96a9ef4e30bb6c3c309919ac00&scope.project.id=194254728c914ec7b65ea0f830811faf&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:28:03.212497 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b85df-5c24-4ddf-a438-c142f60cf563 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:03.213040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b85df-5c24-4ddf-a438-c142f60cf563 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:03.213040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b85df-5c24-4ddf-a438-c142f60cf563 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:03.214461 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 599/1186] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:28:03.219542 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662c66c-e4b9-465e-a894-70385b7dc424 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:03.219984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662c66c-e4b9-465e-a894-70385b7dc424 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:03.220080 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662c66c-e4b9-465e-a894-70385b7dc424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:03.237215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-b662c66c-e4b9-465e-a894-70385b7dc424 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:28:03.238393 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-b662c66c-e4b9-465e-a894-70385b7dc424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:28:03.302463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 588/1187] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:28:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:28:03.325966 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b85df-5c24-4ddf-a438-c142f60cf563 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:28:03.425340 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b85df-5c24-4ddf-a438-c142f60cf563 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:28:03.457477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ce6e3-e178-4b7f-8618-0aa2fca66063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:28:03.458256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ce6e3-e178-4b7f-8618-0aa2fca66063 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:03.458532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ce6e3-e178-4b7f-8618-0aa2fca66063 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:03.458801 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ce6e3-e178-4b7f-8618-0aa2fca66063 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:03.469219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ce6e3-e178-4b7f-8618-0aa2fca66063 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:28:03.477474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-cc2ce6e3-e178-4b7f-8618-0aa2fca66063 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:28:03.477474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:28:03.477474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:28:03.477474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:28:03.477474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:28:03.477474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:28:03.477474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:28:03.477474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:28:03.477474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:28:03.477474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:28:03.477474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:28:03.477474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:28:03.477474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:28:03.477474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:28:03.477474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:28:03.477474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:28:03.478474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:28:03.478474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:28:03.478474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:28:03.478474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:28:03.478474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:28:03.478474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:28:03.478474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:28:03.478474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:28:03.478474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:28:03.478474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:28:03.478474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:28:03.478474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:28:03.478474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:28:03.478474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:28:03.478474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:28:03.478474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:28:03.479368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:28:03.479368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:28:03.479368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:28:03.479368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:28:03.479368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:28:03.479368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:28:03.479368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:28:03.479368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:28:03.479368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:28:03.479368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:28:03.479368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:28:03.479368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:28:03.479368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 600/1188] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:28:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:28:03.484807 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662c66c-e4b9-465e-a894-70385b7dc424 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:28:03.581592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662c66c-e4b9-465e-a894-70385b7dc424 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:28:03.591005 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:28:03.615713 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0cc3ed-b672-4775-bb37-8fa2d7dab9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:28:03.616368 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cc3ed-b672-4775-bb37-8fa2d7dab9f2 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:03.616648 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cc3ed-b672-4775-bb37-8fa2d7dab9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:03.616915 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cc3ed-b672-4775-bb37-8fa2d7dab9f2 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:03.617806 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cc3ed-b672-4775-bb37-8fa2d7dab9f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:28:03.618102 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cc3ed-b672-4775-bb37-8fa2d7dab9f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:28:03.619457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cc3ed-b672-4775-bb37-8fa2d7dab9f2 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:28:03.627246 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 589/1189] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:28:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:28:03.633302 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ce6e3-e178-4b7f-8618-0aa2fca66063 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:28:03.684041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:28:03.733429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ce6e3-e178-4b7f-8618-0aa2fca66063 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:28:03.765865 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d2269-210e-46bd-aa36-a229e2a82b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:28:03.766716 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d2269-210e-46bd-aa36-a229e2a82b3e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:03.766998 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d2269-210e-46bd-aa36-a229e2a82b3e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:03.767273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d2269-210e-46bd-aa36-a229e2a82b3e None admin] PATH_INFO: `/v3/users/cac7cb96a9ef4e30bb6c3c309919ac00` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:03.768451 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d2269-210e-46bd-aa36-a229e2a82b3e None admin] RBAC: Authorizing `identity:get_user(user_id=cac7cb96a9ef4e30bb6c3c309919ac00)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:28:03.769910 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d2269-210e-46bd-aa36-a229e2a82b3e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:28:03.771543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 601/1190] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:28:03 2022] GET /identity/v3/users/cac7cb96a9ef4e30bb6c3c309919ac00 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:28:03.778012 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0cc3ed-b672-4775-bb37-8fa2d7dab9f2 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:28:03.883962 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0cc3ed-b672-4775-bb37-8fa2d7dab9f2 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:28:03.922209 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417850ee-18e6-43fa-9172-be13da054716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:28:03.923069 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417850ee-18e6-43fa-9172-be13da054716 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:03.923343 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417850ee-18e6-43fa-9172-be13da054716 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:03.923615 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417850ee-18e6-43fa-9172-be13da054716 None admin] PATH_INFO: `/v3/projects/194254728c914ec7b65ea0f830811faf` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:03.924803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417850ee-18e6-43fa-9172-be13da054716 None admin] RBAC: Authorizing `identity:get_project(project_id=194254728c914ec7b65ea0f830811faf)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:28:03.926270 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417850ee-18e6-43fa-9172-be13da054716 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:28:03.927983 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 590/1191] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:28:03 2022] GET /identity/v3/projects/194254728c914ec7b65ea0f830811faf => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:28:03.934136 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d2269-210e-46bd-aa36-a229e2a82b3e None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:28:04.038991 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d2269-210e-46bd-aa36-a229e2a82b3e None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:28:04.055849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:28:04.074622 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc12cf-0635-4dfc-aa0b-0d3f38d210a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:28:04.075628 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc12cf-0635-4dfc-aa0b-0d3f38d210a8 None admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:04.075938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc12cf-0635-4dfc-aa0b-0d3f38d210a8 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:04.076268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc12cf-0635-4dfc-aa0b-0d3f38d210a8 None admin] PATH_INFO: `/v3/projects/194254728c914ec7b65ea0f830811faf/users/cac7cb96a9ef4e30bb6c3c309919ac00/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:04.078307 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc12cf-0635-4dfc-aa0b-0d3f38d210a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=194254728c914ec7b65ea0f830811faf, user_id=cac7cb96a9ef4e30bb6c3c309919ac00, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:28:04.079790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc12cf-0635-4dfc-aa0b-0d3f38d210a8 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:28:04.094285 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 602/1192] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:28:03 2022] PUT /identity/v3/projects/194254728c914ec7b65ea0f830811faf/users/cac7cb96a9ef4e30bb6c3c309919ac00/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:28:06.107739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd444836-efde-42ab-b06c-21ae3e159f2d None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:06.108330 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd444836-efde-42ab-b06c-21ae3e159f2d None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:06.108330 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd444836-efde-42ab-b06c-21ae3e159f2d None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:06.109666 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 591/1193] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:28:06.114785 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c095d2-9f26-4c07-93d8-eff2566e0c62 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:06.115197 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c095d2-9f26-4c07-93d8-eff2566e0c62 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:06.115344 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c095d2-9f26-4c07-93d8-eff2566e0c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:06.132264 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-a0c095d2-9f26-4c07-93d8-eff2566e0c62 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:28:06.133498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a0c095d2-9f26-4c07-93d8-eff2566e0c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:28:06.200420 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 603/1194] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:28:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:28:06.224020 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd444836-efde-42ab-b06c-21ae3e159f2d None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:28:06.266191 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:28:06.327076 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd444836-efde-42ab-b06c-21ae3e159f2d None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:28:06.361535 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce58f5-7950-4e11-85eb-fc57b5457035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:28:06.362247 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce58f5-7950-4e11-85eb-fc57b5457035 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:06.362533 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce58f5-7950-4e11-85eb-fc57b5457035 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:06.362809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce58f5-7950-4e11-85eb-fc57b5457035 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:06.374487 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce58f5-7950-4e11-85eb-fc57b5457035 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:28:06.382085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-74ce58f5-7950-4e11-85eb-fc57b5457035 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:28:06.382085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:28:06.382085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:28:06.382085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:28:06.382085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:28:06.382085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:28:06.382085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:28:06.382085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:28:06.382085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:28:06.382085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:28:06.382085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:28:06.382085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:28:06.382085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:28:06.382085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:28:06.382085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:28:06.382085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:28:06.383842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:28:06.383842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:28:06.383842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:28:06.383842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:28:06.383842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:28:06.383842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:28:06.383842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:28:06.383842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:28:06.383842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:28:06.383842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:28:06.383842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:28:06.383842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:28:06.383842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:28:06.383842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:28:06.383842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:28:06.383842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:28:06.385495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:28:06.385495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:28:06.385495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:28:06.385495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:28:06.385495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:28:06.385495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:28:06.385495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:28:06.385495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:28:06.385495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:28:06.385495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:28:06.385495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:28:06.385495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:28:06.385495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 592/1195] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:28:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:28:06.389435 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c095d2-9f26-4c07-93d8-eff2566e0c62 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:28:06.489055 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c095d2-9f26-4c07-93d8-eff2566e0c62 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:28:06.498487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:28:06.525770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ffcd4-9d36-4c25-9025-488e464691c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:28:06.526429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ffcd4-9d36-4c25-9025-488e464691c4 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:06.526714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ffcd4-9d36-4c25-9025-488e464691c4 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:06.526994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ffcd4-9d36-4c25-9025-488e464691c4 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:06.527739 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ffcd4-9d36-4c25-9025-488e464691c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:28:06.528056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ffcd4-9d36-4c25-9025-488e464691c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:28:06.529449 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ffcd4-9d36-4c25-9025-488e464691c4 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:28:06.537344 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 604/1196] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:28:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:28:06.543787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce58f5-7950-4e11-85eb-fc57b5457035 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:28:06.636358 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:28:06.640292 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce58f5-7950-4e11-85eb-fc57b5457035 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:28:06.674527 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3619c2-6f6b-43c9-aad8-4d7bc2c414e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:28:06.675384 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3619c2-6f6b-43c9-aad8-4d7bc2c414e2 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:06.675665 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3619c2-6f6b-43c9-aad8-4d7bc2c414e2 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:06.675940 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3619c2-6f6b-43c9-aad8-4d7bc2c414e2 None admin] PATH_INFO: `/v3/users/cac7cb96a9ef4e30bb6c3c309919ac00` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:06.677163 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3619c2-6f6b-43c9-aad8-4d7bc2c414e2 None admin] RBAC: Authorizing `identity:get_user(user_id=cac7cb96a9ef4e30bb6c3c309919ac00)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:28:06.678584 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3619c2-6f6b-43c9-aad8-4d7bc2c414e2 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:28:06.680263 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 593/1197] 10.222.0.71 () {56 vars in 1177 bytes} [Thu Jul 21 20:28:06 2022] GET /identity/v3/users/cac7cb96a9ef4e30bb6c3c309919ac00 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:28:06.686186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ffcd4-9d36-4c25-9025-488e464691c4 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:28:06.784432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ffcd4-9d36-4c25-9025-488e464691c4 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:28:06.817589 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9cf7d-8be7-45cb-8933-87b2b1a1591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:28:06.818229 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9cf7d-8be7-45cb-8933-87b2b1a1591e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:06.818511 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9cf7d-8be7-45cb-8933-87b2b1a1591e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:06.818786 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9cf7d-8be7-45cb-8933-87b2b1a1591e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:06.858581 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:28:06.886712 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9cf7d-8be7-45cb-8933-87b2b1a1591e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:28:06.888094 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9cf7d-8be7-45cb-8933-87b2b1a1591e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:28:06.984847 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 605/1198] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:28:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:28:06.990822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3619c2-6f6b-43c9-aad8-4d7bc2c414e2 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:28:07.086704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3619c2-6f6b-43c9-aad8-4d7bc2c414e2 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:28:07.119922 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60611420-d28d-4da8-b2c2-367100a1fbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:28:07.120763 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60611420-d28d-4da8-b2c2-367100a1fbee None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:07.121038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60611420-d28d-4da8-b2c2-367100a1fbee None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:07.121333 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60611420-d28d-4da8-b2c2-367100a1fbee None admin] PATH_INFO: `/v3/projects/194254728c914ec7b65ea0f830811faf` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:07.122501 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60611420-d28d-4da8-b2c2-367100a1fbee None admin] RBAC: Authorizing `identity:get_project(project_id=194254728c914ec7b65ea0f830811faf)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:28:07.123910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60611420-d28d-4da8-b2c2-367100a1fbee None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:28:07.125599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 594/1199] 10.222.0.71 () {56 vars in 1186 bytes} [Thu Jul 21 20:28:06 2022] GET /identity/v3/projects/194254728c914ec7b65ea0f830811faf => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:28:07.131945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9cf7d-8be7-45cb-8933-87b2b1a1591e None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:28:07.201401 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:28:07.232793 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9cf7d-8be7-45cb-8933-87b2b1a1591e None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:28:07.266031 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a2813-a9cf-4605-82fc-3e9bb4826008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:28:07.266674 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a2813-a9cf-4605-82fc-3e9bb4826008 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:28:07.266955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a2813-a9cf-4605-82fc-3e9bb4826008 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:28:07.267230 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a2813-a9cf-4605-82fc-3e9bb4826008 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:28:07.268056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a2813-a9cf-4605-82fc-3e9bb4826008 None admin] RBAC: Adding query filter params (role.id=19f5e72c0be44a1197c7288de82e79e8, scope.project.id=194254728c914ec7b65ea0f830811faf, user.id=cac7cb96a9ef4e30bb6c3c309919ac00) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:28:07.268366 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a2813-a9cf-4605-82fc-3e9bb4826008 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:28:07.269815 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a2813-a9cf-4605-82fc-3e9bb4826008 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:28:07.276261 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 606/1200] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:28:07 2022] GET /identity/v3/role_assignments?user.id=cac7cb96a9ef4e30bb6c3c309919ac00&scope.project.id=194254728c914ec7b65ea0f830811faf&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:28:17.115352 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=83 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:28:17.273445 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=33 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:02.961111 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887b288-3670-41e3-83c7-11a57fd35157 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:02.961701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887b288-3670-41e3-83c7-11a57fd35157 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:02.961701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887b288-3670-41e3-83c7-11a57fd35157 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:02.966035 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 595/1201] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:29:02.973387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481785a-dbdd-430f-97ae-7736b7762914 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:02.973884 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481785a-dbdd-430f-97ae-7736b7762914 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:02.973988 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481785a-dbdd-430f-97ae-7736b7762914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:02.994364 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-1481785a-dbdd-430f-97ae-7736b7762914 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:29:02.996314 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-1481785a-dbdd-430f-97ae-7736b7762914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:29:03.089099 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 607/1202] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:29:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:29:03.108982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887b288-3670-41e3-83c7-11a57fd35157 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:03.258826 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887b288-3670-41e3-83c7-11a57fd35157 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:03.312630 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d733bb7-ecd8-4a89-ab39-0830a1087d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:03.313617 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d733bb7-ecd8-4a89-ab39-0830a1087d1c None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:03.313886 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d733bb7-ecd8-4a89-ab39-0830a1087d1c None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:03.314350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d733bb7-ecd8-4a89-ab39-0830a1087d1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:03.330421 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d733bb7-ecd8-4a89-ab39-0830a1087d1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:03.332663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d733bb7-ecd8-4a89-ab39-0830a1087d1c None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:03.362688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-9d733bb7-ecd8-4a89-ab39-0830a1087d1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:03.362688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:03.362688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:03.362688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:03.362688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:03.362688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:03.362688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:03.362688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:03.362688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:03.362688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:03.362688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:03.362688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:03.362688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:29:03.362688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:29:03.362688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:29:03.363918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:29:03.363918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:03.363918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:03.363918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:03.363918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:03.363918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:03.363918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:03.363918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:03.363918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:03.363918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:03.363918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:03.363918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:03.363918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:03.363918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:03.363918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:03.363918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:03.363918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:03.366429 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:29:03.366429 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:29:03.366429 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:03.366429 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:29:03.366429 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 596/1203] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:03.371133 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481785a-dbdd-430f-97ae-7736b7762914 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:03.534465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481785a-dbdd-430f-97ae-7736b7762914 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:03.578218 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:03.588134 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bec3ea-1e70-453d-8e73-021d07776775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:03.588820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bec3ea-1e70-453d-8e73-021d07776775 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:03.589118 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bec3ea-1e70-453d-8e73-021d07776775 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:03.589397 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bec3ea-1e70-453d-8e73-021d07776775 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:03.590137 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bec3ea-1e70-453d-8e73-021d07776775 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:03.590444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bec3ea-1e70-453d-8e73-021d07776775 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:03.591895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bec3ea-1e70-453d-8e73-021d07776775 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:03.608184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 608/1204] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:03.619757 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d733bb7-ecd8-4a89-ab39-0830a1087d1c None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:03.697327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:03.789014 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d733bb7-ecd8-4a89-ab39-0830a1087d1c None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:03.877638 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878fcc7-e6f2-4297-8b49-968e5d0ff909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:03.879319 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878fcc7-e6f2-4297-8b49-968e5d0ff909 None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:03.879989 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878fcc7-e6f2-4297-8b49-968e5d0ff909 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:03.880567 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878fcc7-e6f2-4297-8b49-968e5d0ff909 None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:03.882459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878fcc7-e6f2-4297-8b49-968e5d0ff909 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:03.886336 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878fcc7-e6f2-4297-8b49-968e5d0ff909 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:03.933277 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 597/1205] 10.222.0.71 () {60 vars in 1132 bytes} [Thu Jul 21 20:29:03 2022] POST /identity/v3/users => generated 251 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:29:06.419240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00871d3e-5a06-4204-af8e-0d0fdf398d43 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:06.420067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00871d3e-5a06-4204-af8e-0d0fdf398d43 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:06.420067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00871d3e-5a06-4204-af8e-0d0fdf398d43 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:06.421194 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 609/1206] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:29:06.429820 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f649bf-e812-4562-a5d9-9ef3e2433598 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:06.430366 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f649bf-e812-4562-a5d9-9ef3e2433598 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:06.430463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f649bf-e812-4562-a5d9-9ef3e2433598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:06.448943 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-d4f649bf-e812-4562-a5d9-9ef3e2433598 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:29:06.450300 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d4f649bf-e812-4562-a5d9-9ef3e2433598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:29:06.539103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 598/1207] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:29:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:29:06.564830 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00871d3e-5a06-4204-af8e-0d0fdf398d43 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:06.713185 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00871d3e-5a06-4204-af8e-0d0fdf398d43 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:06.767719 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eab21b-a80a-41e0-b55a-f20d9f246bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:06.768538 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eab21b-a80a-41e0-b55a-f20d9f246bd3 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:06.768987 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eab21b-a80a-41e0-b55a-f20d9f246bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:06.769191 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eab21b-a80a-41e0-b55a-f20d9f246bd3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:06.783427 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eab21b-a80a-41e0-b55a-f20d9f246bd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:29:06.791911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-29eab21b-a80a-41e0-b55a-f20d9f246bd3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:29:06.791911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:06.791911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:06.791911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:06.791911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:06.791911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:06.791911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:06.791911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:06.791911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:06.791911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:06.791911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:06.791911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:06.791911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:29:06.791911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:29:06.791911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:29:06.791911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:29:06.793519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:29:06.793519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:29:06.793519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:29:06.793519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:29:06.793519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:06.793519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:06.793519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:06.793519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:06.793519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:06.793519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:06.793519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:06.793519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:06.793519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:06.793519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:06.793519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:06.793519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:06.794535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:06.794535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:06.794535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:06.794535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:06.794535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:29:06.794535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:29:06.794535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:29:06.794535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:29:06.794535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:29:06.794535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:29:06.794535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:29:06.794535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:06.794535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 610/1208] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:29:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:06.799598 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f649bf-e812-4562-a5d9-9ef3e2433598 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:06.868713 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:06.955479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f649bf-e812-4562-a5d9-9ef3e2433598 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:07.029515 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd55ea4-b8ba-4107-a4d5-34aa7a6e58e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:07.030688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd55ea4-b8ba-4107-a4d5-34aa7a6e58e4 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:07.031122 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd55ea4-b8ba-4107-a4d5-34aa7a6e58e4 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:07.031521 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd55ea4-b8ba-4107-a4d5-34aa7a6e58e4 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:07.032483 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd55ea4-b8ba-4107-a4d5-34aa7a6e58e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:07.032902 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd55ea4-b8ba-4107-a4d5-34aa7a6e58e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:07.035242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd55ea4-b8ba-4107-a4d5-34aa7a6e58e4 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:07.052712 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 599/1209] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:29:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:07.054968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eab21b-a80a-41e0-b55a-f20d9f246bd3 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:07.131796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:07.210921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eab21b-a80a-41e0-b55a-f20d9f246bd3 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:07.252319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83ebac-6ad6-4da4-b15e-3e2bc186340a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:07.252819 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83ebac-6ad6-4da4-b15e-3e2bc186340a None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:07.253126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83ebac-6ad6-4da4-b15e-3e2bc186340a None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:07.253418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83ebac-6ad6-4da4-b15e-3e2bc186340a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:07.269189 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83ebac-6ad6-4da4-b15e-3e2bc186340a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:07.270979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83ebac-6ad6-4da4-b15e-3e2bc186340a None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:07.291465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-ab83ebac-6ad6-4da4-b15e-3e2bc186340a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:07.291465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:07.291465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:07.291465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:07.291465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:07.291465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:07.291465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:07.291465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:07.291465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:07.291465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:07.291465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:07.291465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:07.291465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:29:07.291465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:29:07.291465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:29:07.292790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:29:07.292790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:07.292790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:07.292790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:07.292790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:07.292790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:07.292790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:07.292790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:07.292790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:07.292790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:07.292790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:07.292790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:07.292790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:07.292790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:07.292790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:07.292790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:07.292790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:07.294004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:29:07.294004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:29:07.294004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:07.294004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:07.299061 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 611/1210] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:07.299891 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd55ea4-b8ba-4107-a4d5-34aa7a6e58e4 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:07.446005 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:07.449722 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd55ea4-b8ba-4107-a4d5-34aa7a6e58e4 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:07.512126 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a039d-4d67-48f5-83d4-1be8929a42ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:07.512991 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a039d-4d67-48f5-83d4-1be8929a42ae None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:07.513397 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a039d-4d67-48f5-83d4-1be8929a42ae None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:07.513772 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a039d-4d67-48f5-83d4-1be8929a42ae None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:07.514631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a039d-4d67-48f5-83d4-1be8929a42ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:07.515254 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a039d-4d67-48f5-83d4-1be8929a42ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:07.517215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a039d-4d67-48f5-83d4-1be8929a42ae None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:07.542388 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 600/1211] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:07.547878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83ebac-6ad6-4da4-b15e-3e2bc186340a None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:07.672306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83ebac-6ad6-4da4-b15e-3e2bc186340a None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:07.675566 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:07.715918 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb2276-195f-491f-b130-3fe86cca9c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:07.716757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb2276-195f-491f-b130-3fe86cca9c82 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:07.717035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb2276-195f-491f-b130-3fe86cca9c82 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:07.717343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb2276-195f-491f-b130-3fe86cca9c82 None admin] PATH_INFO: `/v3/users/placement` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:07.727147 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb2276-195f-491f-b130-3fe86cca9c82 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:07.728709 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb2276-195f-491f-b130-3fe86cca9c82 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:07.750306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-39fb2276-195f-491f-b130-3fe86cca9c82 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:29:07.750306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:07.750306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:07.750306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:07.750306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:07.750306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:07.750306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:07.750306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:07.750306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:07.750306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:07.750306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:07.750306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:07.750306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:29:07.750306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:29:07.750306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:29:07.750306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:29:07.752980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:07.752980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:07.752980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:29:07.752980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:29:07.752980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:29:07.752980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:29:07.752980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:07.752980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:07.752980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:07.752980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:07.752980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:07.752980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:07.752980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:07.752980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:07.752980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:07.752980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:07.752980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:07.752980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:07.755652 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:07.755652 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:07.755652 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:29:07.755652 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:29:07.755652 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:29:07.755652 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:29:07.755652 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:29:07.755652 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:29:07.755652 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:29:07.755652 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:07.755652 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 612/1212] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:07.760091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a039d-4d67-48f5-83d4-1be8929a42ae None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:07.903499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a039d-4d67-48f5-83d4-1be8929a42ae None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:07.960733 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6a0e9-78d1-489b-896b-770354c493ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:07.961833 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6a0e9-78d1-489b-896b-770354c493ef None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:07.962201 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6a0e9-78d1-489b-896b-770354c493ef None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:07.962577 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6a0e9-78d1-489b-896b-770354c493ef None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:07.963719 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6a0e9-78d1-489b-896b-770354c493ef None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:07.964187 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6a0e9-78d1-489b-896b-770354c493ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:07.966283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6a0e9-78d1-489b-896b-770354c493ef None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:07.999214 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 601/1213] 10.222.0.71 () {56 vars in 1143 bytes} [Thu Jul 21 20:29:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:08.007664 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb2276-195f-491f-b130-3fe86cca9c82 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:08.154140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb2276-195f-491f-b130-3fe86cca9c82 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:08.204096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a09fe-445a-4afe-91c3-0945d92d31b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:08.204765 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a09fe-445a-4afe-91c3-0945d92d31b2 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:08.205054 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a09fe-445a-4afe-91c3-0945d92d31b2 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:08.205363 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a09fe-445a-4afe-91c3-0945d92d31b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:08.245817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:08.291130 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a09fe-445a-4afe-91c3-0945d92d31b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:08.292815 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a09fe-445a-4afe-91c3-0945d92d31b2 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:08.408579 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 613/1214] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:29:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:29:08.417961 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6a0e9-78d1-489b-896b-770354c493ef None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:08.465503 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:08.601691 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6a0e9-78d1-489b-896b-770354c493ef None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:08.657491 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fd273-1ce4-44db-be4f-325e6e0da667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:08.658887 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fd273-1ce4-44db-be4f-325e6e0da667 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:08.659184 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fd273-1ce4-44db-be4f-325e6e0da667 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:08.659675 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fd273-1ce4-44db-be4f-325e6e0da667 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:08.674821 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fd273-1ce4-44db-be4f-325e6e0da667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:08.676397 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fd273-1ce4-44db-be4f-325e6e0da667 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:08.699704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-341fd273-1ce4-44db-be4f-325e6e0da667 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:08.699704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:08.699704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:08.699704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:08.699704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:08.699704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:08.699704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:08.699704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:08.699704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:08.699704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:08.699704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:08.699704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:08.699704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:29:08.699704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:29:08.699704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:29:08.700685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:29:08.700685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:08.700685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:08.700685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:08.700685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:08.700685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:08.700685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:08.700685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:08.700685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:08.700685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:08.700685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:08.700685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:08.700685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:08.700685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:08.700685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:08.700685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:08.700685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:08.701998 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:29:08.701998 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:29:08.701998 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:08.701998 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:29:08.701998 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 602/1215] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:29:08.709113 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a09fe-445a-4afe-91c3-0945d92d31b2 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:08.784969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:08.831559 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a09fe-445a-4afe-91c3-0945d92d31b2 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:08.874144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a458c77-02b2-44d5-b1a9-9cb8ccfe8766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:08.874986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a458c77-02b2-44d5-b1a9-9cb8ccfe8766 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:08.875383 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a458c77-02b2-44d5-b1a9-9cb8ccfe8766 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:08.875767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a458c77-02b2-44d5-b1a9-9cb8ccfe8766 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:08.876616 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a458c77-02b2-44d5-b1a9-9cb8ccfe8766 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:08.877028 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a458c77-02b2-44d5-b1a9-9cb8ccfe8766 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:08.878548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a458c77-02b2-44d5-b1a9-9cb8ccfe8766 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:08.894818 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 614/1216] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:08.901490 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fd273-1ce4-44db-be4f-325e6e0da667 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:08.993092 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:09.043402 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fd273-1ce4-44db-be4f-325e6e0da667 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:09.091827 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece2d87-3548-4b2c-890e-6d4f79f16d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:09.092793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece2d87-3548-4b2c-890e-6d4f79f16d44 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:09.093152 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece2d87-3548-4b2c-890e-6d4f79f16d44 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:09.093499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece2d87-3548-4b2c-890e-6d4f79f16d44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:09.101867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece2d87-3548-4b2c-890e-6d4f79f16d44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:09.103413 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece2d87-3548-4b2c-890e-6d4f79f16d44 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:09.117878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-cece2d87-3548-4b2c-890e-6d4f79f16d44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:29:09.117878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:09.117878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:09.117878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:09.117878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:09.117878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:09.117878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:09.117878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:09.117878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:09.117878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:09.117878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:09.117878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:09.117878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:29:09.117878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:29:09.117878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:29:09.119420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:29:09.119420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:09.119420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:09.119420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:09.119420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:09.119420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:09.119420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:09.119420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:09.119420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:09.119420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:09.119420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:09.119420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:09.119420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:09.119420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:09.119420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:09.119420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:09.119420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:09.120637 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:29:09.120637 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:29:09.120637 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:29:09.120637 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:29:09.120637 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:29:09.120637 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:29:09.120637 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:29:09.120637 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:29:09.120637 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 603/1217] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:29:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:09.127977 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a458c77-02b2-44d5-b1a9-9cb8ccfe8766 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:09.249010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a458c77-02b2-44d5-b1a9-9cb8ccfe8766 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:09.275235 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:09.289645 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663fd7f-f935-4402-9269-32325265dde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:09.290406 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663fd7f-f935-4402-9269-32325265dde7 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:09.290751 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663fd7f-f935-4402-9269-32325265dde7 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:09.291114 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663fd7f-f935-4402-9269-32325265dde7 None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:09.292080 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2663fd7f-f935-4402-9269-32325265dde7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:09.292382 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2663fd7f-f935-4402-9269-32325265dde7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:09.293967 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2663fd7f-f935-4402-9269-32325265dde7 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:09.307372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 615/1218] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:29:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:09.313771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece2d87-3548-4b2c-890e-6d4f79f16d44 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:09.446241 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece2d87-3548-4b2c-890e-6d4f79f16d44 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:09.490691 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:09.499656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce026d0-2224-4966-ac3e-8b5ed5b606e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:09.500313 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce026d0-2224-4966-ac3e-8b5ed5b606e6 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:09.500600 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce026d0-2224-4966-ac3e-8b5ed5b606e6 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:09.500940 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce026d0-2224-4966-ac3e-8b5ed5b606e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:09.502015 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce026d0-2224-4966-ac3e-8b5ed5b606e6 None admin] RBAC: Adding query filter params (role.id=9a48ab1bf32a45828e9e73599c980dac, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=364e80ec79b94812a1df76c62c2b13a0) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:09.503218 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce026d0-2224-4966-ac3e-8b5ed5b606e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:09.504024 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce026d0-2224-4966-ac3e-8b5ed5b606e6 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:09.515994 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 604/1219] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:29:09 2022] GET /identity/v3/role_assignments?user.id=364e80ec79b94812a1df76c62c2b13a0&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=9a48ab1bf32a45828e9e73599c980dac => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:11.853991 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1077c-7c42-451d-aa5a-364073fbaaf9 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:11.854772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1077c-7c42-451d-aa5a-364073fbaaf9 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:11.854871 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1077c-7c42-451d-aa5a-364073fbaaf9 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:11.857415 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 616/1220] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:29:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:29:11.866583 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7ee17-421d-4c0e-9db4-84dffd53bcb1 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:11.867182 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7ee17-421d-4c0e-9db4-84dffd53bcb1 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:11.867331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7ee17-421d-4c0e-9db4-84dffd53bcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:11.887672 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-dad7ee17-421d-4c0e-9db4-84dffd53bcb1 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:29:11.888924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-dad7ee17-421d-4c0e-9db4-84dffd53bcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:29:11.982365 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 605/1221] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:29:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:29:12.008385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1077c-7c42-451d-aa5a-364073fbaaf9 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:12.141607 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1077c-7c42-451d-aa5a-364073fbaaf9 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:12.179082 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30e838-1f7e-4a19-b45f-a96708aefa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:12.179879 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30e838-1f7e-4a19-b45f-a96708aefa94 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:12.180167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30e838-1f7e-4a19-b45f-a96708aefa94 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:12.180434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30e838-1f7e-4a19-b45f-a96708aefa94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:12.192008 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30e838-1f7e-4a19-b45f-a96708aefa94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:29:12.200582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-3f30e838-1f7e-4a19-b45f-a96708aefa94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:29:12.200582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:12.200582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:12.200582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:12.200582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:12.200582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:12.200582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:12.200582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:12.200582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:12.200582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:12.200582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:12.200582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:12.200582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:29:12.200582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:29:12.200582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:29:12.200582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:29:12.202527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:29:12.202527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:29:12.202527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:29:12.202527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:29:12.202527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:12.202527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:12.202527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:12.202527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:12.202527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:12.202527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:12.202527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:12.202527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:12.202527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:12.202527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:12.202527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:12.202527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:12.207047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:12.207047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:12.207047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:12.207047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:12.207047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:29:12.207047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:29:12.207047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:29:12.207047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:29:12.207047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:29:12.207047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:29:12.207047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:29:12.207047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:12.207047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 617/1222] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:29:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:12.216755 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7ee17-421d-4c0e-9db4-84dffd53bcb1 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:12.334884 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7ee17-421d-4c0e-9db4-84dffd53bcb1 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:12.362284 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:12.378539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5472ea-0025-4e2a-b73c-16301a650a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:12.379251 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5472ea-0025-4e2a-b73c-16301a650a60 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:12.379528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5472ea-0025-4e2a-b73c-16301a650a60 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:12.379864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5472ea-0025-4e2a-b73c-16301a650a60 None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:12.380754 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5472ea-0025-4e2a-b73c-16301a650a60 None admin] RBAC: Adding query filter params (name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:12.381274 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5472ea-0025-4e2a-b73c-16301a650a60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:12.382808 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5472ea-0025-4e2a-b73c-16301a650a60 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:12.392712 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 606/1223] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:29:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:12.400449 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30e838-1f7e-4a19-b45f-a96708aefa94 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:12.454080 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:12.522460 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30e838-1f7e-4a19-b45f-a96708aefa94 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:12.564693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43992a9-7d62-4c35-be94-013873bc5adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:12.565579 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43992a9-7d62-4c35-be94-013873bc5adc None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:12.565827 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43992a9-7d62-4c35-be94-013873bc5adc None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:12.566104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43992a9-7d62-4c35-be94-013873bc5adc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:12.577877 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43992a9-7d62-4c35-be94-013873bc5adc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:12.579346 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43992a9-7d62-4c35-be94-013873bc5adc None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:12.598259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-f43992a9-7d62-4c35-be94-013873bc5adc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:12.598259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:12.598259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:12.598259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:12.598259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:12.598259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:12.598259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:12.598259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:12.598259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:12.598259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:12.598259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:12.598259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:12.598259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:29:12.598259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:29:12.598259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:29:12.600360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:29:12.600360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:12.600360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:12.600360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:12.600360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:12.600360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:12.600360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:12.600360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:12.600360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:12.600360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:12.600360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:12.600360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:12.600360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:12.600360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:12.600360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:12.600360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:12.600360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:12.601512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:29:12.601512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:29:12.601512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:12.601512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:12.601512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 618/1224] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:12.606018 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5472ea-0025-4e2a-b73c-16301a650a60 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:12.731670 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5472ea-0025-4e2a-b73c-16301a650a60 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:12.775509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed1692-1588-4c43-9280-d8581aa4e2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:12.776273 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed1692-1588-4c43-9280-d8581aa4e2be None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:12.776690 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed1692-1588-4c43-9280-d8581aa4e2be None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:12.776918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed1692-1588-4c43-9280-d8581aa4e2be None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:12.777788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed1692-1588-4c43-9280-d8581aa4e2be None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:12.778099 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed1692-1588-4c43-9280-d8581aa4e2be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:12.779502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed1692-1588-4c43-9280-d8581aa4e2be None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:12.793437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 607/1225] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:12.799828 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43992a9-7d62-4c35-be94-013873bc5adc None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:12.911350 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:12.919683 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43992a9-7d62-4c35-be94-013873bc5adc None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:12.962446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6d0f6-0d85-4993-bb5e-5dd4a94b18b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:12.963838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6d0f6-0d85-4993-bb5e-5dd4a94b18b9 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:12.964440 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6d0f6-0d85-4993-bb5e-5dd4a94b18b9 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:12.964882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6d0f6-0d85-4993-bb5e-5dd4a94b18b9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:12.977633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd6d0f6-0d85-4993-bb5e-5dd4a94b18b9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:12.979175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd6d0f6-0d85-4993-bb5e-5dd4a94b18b9 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:12.994270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-1bd6d0f6-0d85-4993-bb5e-5dd4a94b18b9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:29:12.994270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:12.994270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:12.994270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:12.994270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:12.994270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:12.994270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:12.994270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:12.994270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:12.994270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:12.994270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:12.994270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:12.994270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:29:12.994270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:29:12.994270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:29:12.994270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:29:12.995408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:12.995408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:12.995408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:29:12.995408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:29:12.995408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:29:12.995408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:29:12.995408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:12.995408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:12.995408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:12.995408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:12.995408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:12.995408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:12.995408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:12.995408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:12.995408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:12.995408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:12.995408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:12.995408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:12.997172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:12.997172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:12.997172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:29:12.997172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:29:12.997172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:29:12.997172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:29:12.997172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:29:12.997172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:29:12.997172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:29:12.997172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:13.000265 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 619/1226] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:13.001778 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed1692-1588-4c43-9280-d8581aa4e2be None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:13.071444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:13.136901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed1692-1588-4c43-9280-d8581aa4e2be None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:13.179689 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86f557-0ed1-456c-8fb8-58351e14caa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:13.180425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86f557-0ed1-456c-8fb8-58351e14caa8 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:13.180711 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86f557-0ed1-456c-8fb8-58351e14caa8 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:13.180985 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86f557-0ed1-456c-8fb8-58351e14caa8 None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:13.181873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86f557-0ed1-456c-8fb8-58351e14caa8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:13.182183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86f557-0ed1-456c-8fb8-58351e14caa8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:13.183686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86f557-0ed1-456c-8fb8-58351e14caa8 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:13.207643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 608/1227] 10.222.0.71 () {56 vars in 1143 bytes} [Thu Jul 21 20:29:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:13.214497 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6d0f6-0d85-4993-bb5e-5dd4a94b18b9 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:13.335149 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6d0f6-0d85-4993-bb5e-5dd4a94b18b9 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:13.395158 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2d636-097c-4fe7-897d-067cfb838a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:13.398172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2d636-097c-4fe7-897d-067cfb838a2f None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:13.398684 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2d636-097c-4fe7-897d-067cfb838a2f None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:13.399181 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2d636-097c-4fe7-897d-067cfb838a2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:13.422868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f2d636-097c-4fe7-897d-067cfb838a2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:13.427101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f2d636-097c-4fe7-897d-067cfb838a2f None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:13.462897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-33f2d636-097c-4fe7-897d-067cfb838a2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:13.462897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:13.462897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:13.462897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:13.462897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:13.462897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:13.462897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:13.462897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:13.462897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:13.462897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:13.462897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:13.462897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:13.462897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:29:13.462897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:29:13.462897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:29:13.464063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:29:13.464063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:13.464063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:13.464063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:13.464063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:13.464063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:13.464063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:13.464063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:13.464063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:13.464063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:13.464063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:13.464063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:13.464063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:13.464063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:13.464063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:13.464063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:13.464063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:13.465055 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:29:13.465055 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:29:13.465055 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:13.465055 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:13.465055 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 620/1228] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:13.472803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86f557-0ed1-456c-8fb8-58351e14caa8 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:13.610643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86f557-0ed1-456c-8fb8-58351e14caa8 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:13.613004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:13.653182 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0da8f-ea84-4022-a495-4212da87f3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:13.654259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0da8f-ea84-4022-a495-4212da87f3fd None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:13.654589 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0da8f-ea84-4022-a495-4212da87f3fd None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:13.654854 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0da8f-ea84-4022-a495-4212da87f3fd None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:13.655805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0da8f-ea84-4022-a495-4212da87f3fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:13.656178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0da8f-ea84-4022-a495-4212da87f3fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:13.657854 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0da8f-ea84-4022-a495-4212da87f3fd None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:13.671531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 609/1229] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:13.679113 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2d636-097c-4fe7-897d-067cfb838a2f None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:13.689117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:13.793240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2d636-097c-4fe7-897d-067cfb838a2f None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:13.845868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26364613-14dc-4c0a-9876-805a91a30ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:13.846858 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26364613-14dc-4c0a-9876-805a91a30ef2 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:13.847255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26364613-14dc-4c0a-9876-805a91a30ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:13.847517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26364613-14dc-4c0a-9876-805a91a30ef2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:13.855307 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26364613-14dc-4c0a-9876-805a91a30ef2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:13.856825 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26364613-14dc-4c0a-9876-805a91a30ef2 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:13.872854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-26364613-14dc-4c0a-9876-805a91a30ef2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:29:13.872854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:13.872854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:13.872854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:13.872854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:13.872854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:13.872854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:13.872854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:13.872854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:13.872854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:13.872854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:13.872854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:13.872854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:29:13.872854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:29:13.872854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:29:13.874368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:29:13.874368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:13.874368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:13.874368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:13.874368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:13.874368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:13.874368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:13.874368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:13.874368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:13.874368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:13.874368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:13.874368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:13.874368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:13.874368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:13.874368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:13.874368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:13.874368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:13.875560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:29:13.875560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:29:13.875560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:29:13.875560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:29:13.875560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:29:13.875560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:29:13.875560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:29:13.875560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:13.875560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 621/1230] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:29:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:29:13.880165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0da8f-ea84-4022-a495-4212da87f3fd None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:14.010695 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0da8f-ea84-4022-a495-4212da87f3fd None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:14.059491 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf24417-d5c5-4a93-ab46-49295085cfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:14.060336 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf24417-d5c5-4a93-ab46-49295085cfcd None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:14.060621 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf24417-d5c5-4a93-ab46-49295085cfcd None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:14.060897 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf24417-d5c5-4a93-ab46-49295085cfcd None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:14.061929 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf24417-d5c5-4a93-ab46-49295085cfcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:14.062283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf24417-d5c5-4a93-ab46-49295085cfcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:14.063816 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf24417-d5c5-4a93-ab46-49295085cfcd None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:14.078373 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 610/1231] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:29:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:29:14.085019 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26364613-14dc-4c0a-9876-805a91a30ef2 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:14.162786 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:14.207666 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26364613-14dc-4c0a-9876-805a91a30ef2 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:14.248523 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4200f-469a-407c-a6d1-b77ab4dbcf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:14.250716 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4200f-469a-407c-a6d1-b77ab4dbcf08 None admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:14.250716 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4200f-469a-407c-a6d1-b77ab4dbcf08 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:14.250716 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4200f-469a-407c-a6d1-b77ab4dbcf08 None admin] PATH_INFO: `/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/364e80ec79b94812a1df76c62c2b13a0/roles/9a48ab1bf32a45828e9e73599c980dac` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:14.267290 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4200f-469a-407c-a6d1-b77ab4dbcf08 None admin] RBAC: Authorizing `identity:create_grant(project_id=37e4afa595894a9994a8e6981ebb7c41, user_id=364e80ec79b94812a1df76c62c2b13a0, role_id=9a48ab1bf32a45828e9e73599c980dac)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:14.268763 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4200f-469a-407c-a6d1-b77ab4dbcf08 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:14.287579 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 622/1232] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:29:14 2022] PUT /identity/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/364e80ec79b94812a1df76c62c2b13a0/roles/9a48ab1bf32a45828e9e73599c980dac => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:29:16.586040 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebb463-4cb6-47b7-b2cf-d3204cfb068c None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:16.586906 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebb463-4cb6-47b7-b2cf-d3204cfb068c None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:16.586906 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebb463-4cb6-47b7-b2cf-d3204cfb068c None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:16.588037 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 611/1233] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:29:16.596165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9e943-8915-4a54-b1f5-d74ed0ccfb5f None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:16.597001 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9e943-8915-4a54-b1f5-d74ed0ccfb5f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:16.597166 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9e943-8915-4a54-b1f5-d74ed0ccfb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:16.618816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-60a9e943-8915-4a54-b1f5-d74ed0ccfb5f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:29:16.620067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-60a9e943-8915-4a54-b1f5-d74ed0ccfb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:29:16.729221 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 623/1234] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:29:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:29:16.758597 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ebb463-4cb6-47b7-b2cf-d3204cfb068c None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:16.805703 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:16.923975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ebb463-4cb6-47b7-b2cf-d3204cfb068c None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:16.981888 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bdd17-691f-48c9-8809-21fe0b08726e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:16.983325 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bdd17-691f-48c9-8809-21fe0b08726e None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:16.983800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bdd17-691f-48c9-8809-21fe0b08726e None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:16.984304 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bdd17-691f-48c9-8809-21fe0b08726e None admin] PATH_INFO: `/v3/roles/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:17.005142 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118bdd17-691f-48c9-8809-21fe0b08726e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:29:17.014791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-118bdd17-691f-48c9-8809-21fe0b08726e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:29:17.014791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:17.014791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:17.014791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:17.014791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:17.014791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:17.014791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:17.014791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:17.014791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:17.014791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:17.014791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:17.014791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:17.014791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:29:17.014791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:29:17.014791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:29:17.014791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:29:17.016825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:29:17.016825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:29:17.016825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:29:17.016825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:29:17.016825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:17.016825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:17.016825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:17.016825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:17.016825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:17.016825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:17.016825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:17.016825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:17.016825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:17.016825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:17.016825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:17.016825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:17.018646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:17.018646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:17.018646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:17.018646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:17.018646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:29:17.018646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:29:17.018646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:29:17.018646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:29:17.018646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:29:17.018646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:29:17.018646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:29:17.018646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:29:17.018646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 612/1235] 10.222.0.71 () {56 vars in 1102 bytes} [Thu Jul 21 20:29:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:17.027290 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9e943-8915-4a54-b1f5-d74ed0ccfb5f None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:17.095514 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:17.168810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9e943-8915-4a54-b1f5-d74ed0ccfb5f None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:17.214388 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396318ae-1d5c-415d-8a98-a8895603dbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:17.215257 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396318ae-1d5c-415d-8a98-a8895603dbfc None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:17.215588 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396318ae-1d5c-415d-8a98-a8895603dbfc None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:17.215899 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396318ae-1d5c-415d-8a98-a8895603dbfc None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:17.216907 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396318ae-1d5c-415d-8a98-a8895603dbfc None admin] RBAC: Adding query filter params (name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:17.217268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396318ae-1d5c-415d-8a98-a8895603dbfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:17.219109 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396318ae-1d5c-415d-8a98-a8895603dbfc None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:17.233190 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 624/1236] 10.222.0.71 () {56 vars in 1103 bytes} [Thu Jul 21 20:29:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:29:17.240868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bdd17-691f-48c9-8809-21fe0b08726e None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:17.360127 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:17.383962 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bdd17-691f-48c9-8809-21fe0b08726e None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:17.436574 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab3f07-d2ba-4266-b3ff-b093f4a4f05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:17.437441 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab3f07-d2ba-4266-b3ff-b093f4a4f05e None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:17.437721 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab3f07-d2ba-4266-b3ff-b093f4a4f05e None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:17.437996 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab3f07-d2ba-4266-b3ff-b093f4a4f05e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:17.450853 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab3f07-d2ba-4266-b3ff-b093f4a4f05e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:17.455108 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab3f07-d2ba-4266-b3ff-b093f4a4f05e None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:17.473317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-ebab3f07-d2ba-4266-b3ff-b093f4a4f05e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:17.473317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:17.473317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:17.473317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:17.473317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:17.473317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:17.473317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:17.473317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:17.473317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:17.473317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:17.473317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:17.473317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:17.473317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:29:17.473317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:29:17.473317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:29:17.475137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:29:17.475137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:17.475137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:17.475137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:17.475137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:17.475137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:17.475137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:17.475137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:17.475137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:17.475137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:17.475137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:17.475137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:17.475137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:17.475137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:17.475137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:17.475137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:17.475137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:17.484150 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:29:17.484150 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:29:17.484150 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:17.484150 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:29:17.484150 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 613/1237] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:17.484499 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396318ae-1d5c-415d-8a98-a8895603dbfc None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:17.620449 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396318ae-1d5c-415d-8a98-a8895603dbfc None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:17.634020 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:17.671151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0600a-8e97-4b2d-a8b4-c5ea7c9ad0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:17.672366 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0600a-8e97-4b2d-a8b4-c5ea7c9ad0aa None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:17.672728 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0600a-8e97-4b2d-a8b4-c5ea7c9ad0aa None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:17.673109 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0600a-8e97-4b2d-a8b4-c5ea7c9ad0aa None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:17.674721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0600a-8e97-4b2d-a8b4-c5ea7c9ad0aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:17.675086 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0600a-8e97-4b2d-a8b4-c5ea7c9ad0aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:17.676685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0600a-8e97-4b2d-a8b4-c5ea7c9ad0aa None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:17.695534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 625/1238] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:17.702378 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab3f07-d2ba-4266-b3ff-b093f4a4f05e None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:17.829401 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab3f07-d2ba-4266-b3ff-b093f4a4f05e None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:17.877341 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653da60b-6400-4474-8d03-eb1cecf473a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:17.878327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653da60b-6400-4474-8d03-eb1cecf473a4 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:17.878943 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653da60b-6400-4474-8d03-eb1cecf473a4 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:17.879282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653da60b-6400-4474-8d03-eb1cecf473a4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:17.884420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:17.895215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653da60b-6400-4474-8d03-eb1cecf473a4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:17.897231 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653da60b-6400-4474-8d03-eb1cecf473a4 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:17.920709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-653da60b-6400-4474-8d03-eb1cecf473a4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:29:17.920709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:17.920709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:17.920709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:17.920709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:17.920709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:17.920709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:17.920709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:17.920709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:17.920709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:17.920709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:17.920709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:17.920709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:29:17.920709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:29:17.920709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:29:17.920709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:29:17.922727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:17.922727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:17.922727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:29:17.922727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:29:17.922727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:29:17.922727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:29:17.922727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:17.922727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:17.922727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:17.922727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:17.922727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:17.922727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:17.922727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:17.922727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:17.922727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:17.922727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:17.922727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:17.922727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:17.924631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:17.924631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:17.924631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:29:17.924631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:29:17.924631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:29:17.924631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:29:17.924631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:29:17.924631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:29:17.924631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:29:17.924631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:29:17.924631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 614/1239] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:29:17.932578 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0600a-8e97-4b2d-a8b4-c5ea7c9ad0aa None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:18.122366 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0600a-8e97-4b2d-a8b4-c5ea7c9ad0aa None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:18.178181 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9be977-f6fe-410c-9ef1-66473e0b76c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:18.179055 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9be977-f6fe-410c-9ef1-66473e0b76c2 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:18.179438 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9be977-f6fe-410c-9ef1-66473e0b76c2 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:18.179834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9be977-f6fe-410c-9ef1-66473e0b76c2 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:18.181217 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9be977-f6fe-410c-9ef1-66473e0b76c2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:18.181621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9be977-f6fe-410c-9ef1-66473e0b76c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:18.183649 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9be977-f6fe-410c-9ef1-66473e0b76c2 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:18.209643 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 626/1240] 10.222.0.71 () {56 vars in 1143 bytes} [Thu Jul 21 20:29:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:18.217206 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653da60b-6400-4474-8d03-eb1cecf473a4 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:18.355221 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653da60b-6400-4474-8d03-eb1cecf473a4 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:18.397166 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e959c9-26f9-4bd6-be1a-f2264ebd26d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:18.397742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e959c9-26f9-4bd6-be1a-f2264ebd26d8 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:18.398013 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e959c9-26f9-4bd6-be1a-f2264ebd26d8 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:18.398295 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e959c9-26f9-4bd6-be1a-f2264ebd26d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:18.487487 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:18.502223 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e959c9-26f9-4bd6-be1a-f2264ebd26d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:18.504535 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e959c9-26f9-4bd6-be1a-f2264ebd26d8 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:18.665984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 615/1241] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:29:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:29:18.673423 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9be977-f6fe-410c-9ef1-66473e0b76c2 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:18.708045 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:18.809846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9be977-f6fe-410c-9ef1-66473e0b76c2 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:18.859138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8024c6-e9d9-4059-a054-426b235cf550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:18.860184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8024c6-e9d9-4059-a054-426b235cf550 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:18.860630 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8024c6-e9d9-4059-a054-426b235cf550 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:18.861062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8024c6-e9d9-4059-a054-426b235cf550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:18.878552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8024c6-e9d9-4059-a054-426b235cf550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:18.881190 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8024c6-e9d9-4059-a054-426b235cf550 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:18.904241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-3e8024c6-e9d9-4059-a054-426b235cf550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:18.904241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:18.904241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:18.904241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:18.904241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:18.904241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:18.904241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:18.904241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:18.904241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:18.904241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:18.904241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:18.904241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:18.904241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:29:18.904241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:29:18.904241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:29:18.905860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:29:18.905860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:18.905860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:18.905860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:18.905860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:18.905860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:18.905860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:18.905860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:18.905860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:18.905860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:18.905860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:18.905860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:18.905860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:18.905860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:18.905860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:18.905860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:18.905860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:18.906879 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:29:18.906879 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:29:18.906879 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:18.906879 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:18.906879 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 627/1242] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:29:18.913627 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e959c9-26f9-4bd6-be1a-f2264ebd26d8 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:19.056294 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:19.069016 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e959c9-26f9-4bd6-be1a-f2264ebd26d8 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:19.130912 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b9566-7b75-4406-8050-5ec4c9b05e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:19.131833 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b9566-7b75-4406-8050-5ec4c9b05e34 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:19.132276 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b9566-7b75-4406-8050-5ec4c9b05e34 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:19.132741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b9566-7b75-4406-8050-5ec4c9b05e34 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:19.134155 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b9566-7b75-4406-8050-5ec4c9b05e34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:19.134689 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b9566-7b75-4406-8050-5ec4c9b05e34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:19.137218 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b9566-7b75-4406-8050-5ec4c9b05e34 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:19.157559 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 616/1243] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:29:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:19.167493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8024c6-e9d9-4059-a054-426b235cf550 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:19.271691 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:19.308864 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8024c6-e9d9-4059-a054-426b235cf550 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:19.354516 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f59d58-ec83-4554-91c3-03502645e051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:19.357000 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f59d58-ec83-4554-91c3-03502645e051 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:19.357446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f59d58-ec83-4554-91c3-03502645e051 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:19.357895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f59d58-ec83-4554-91c3-03502645e051 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:19.373942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f59d58-ec83-4554-91c3-03502645e051 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:19.375750 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f59d58-ec83-4554-91c3-03502645e051 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:19.395510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-03f59d58-ec83-4554-91c3-03502645e051 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:29:19.395510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:19.395510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:19.395510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:19.395510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:19.395510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:19.395510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:19.395510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:19.395510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:19.395510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:19.395510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:19.395510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:19.395510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:29:19.395510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:29:19.395510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:29:19.398327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:29:19.398327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:19.398327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:19.398327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:19.398327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:19.398327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:19.398327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:19.398327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:19.398327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:19.398327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:19.398327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:19.398327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:19.398327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:19.398327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:19.398327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:19.398327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:19.398327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:19.400165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:29:19.400165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:29:19.400165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:29:19.400165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:29:19.400165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:29:19.400165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:29:19.400165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:29:19.400165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:19.400165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 628/1244] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:29:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:29:19.406887 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b9566-7b75-4406-8050-5ec4c9b05e34 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:19.553562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b9566-7b75-4406-8050-5ec4c9b05e34 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:19.596355 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62487981-581d-4758-bbad-2ad080603f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:19.596953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62487981-581d-4758-bbad-2ad080603f5a None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:19.597264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62487981-581d-4758-bbad-2ad080603f5a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:19.597621 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62487981-581d-4758-bbad-2ad080603f5a None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:19.598350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62487981-581d-4758-bbad-2ad080603f5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:19.598660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62487981-581d-4758-bbad-2ad080603f5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:19.600215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62487981-581d-4758-bbad-2ad080603f5a None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:19.615235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 617/1245] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:29:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:19.621940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f59d58-ec83-4554-91c3-03502645e051 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:19.773148 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f59d58-ec83-4554-91c3-03502645e051 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:19.823824 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a419c4-b027-4d69-ba6b-c423387687d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:19.824736 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a419c4-b027-4d69-ba6b-c423387687d6 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:19.825220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a419c4-b027-4d69-ba6b-c423387687d6 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:19.825664 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a419c4-b027-4d69-ba6b-c423387687d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:19.826826 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a419c4-b027-4d69-ba6b-c423387687d6 None admin] RBAC: Adding query filter params (role.id=9a48ab1bf32a45828e9e73599c980dac, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=364e80ec79b94812a1df76c62c2b13a0) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:19.827227 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a419c4-b027-4d69-ba6b-c423387687d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:19.828973 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a419c4-b027-4d69-ba6b-c423387687d6 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:19.837240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 629/1246] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:29:19 2022] GET /identity/v3/role_assignments?user.id=364e80ec79b94812a1df76c62c2b13a0&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=9a48ab1bf32a45828e9e73599c980dac => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:29:22.158487 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e95bf-5c78-4faa-8d45-e9549b82cf38 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:22.159322 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e95bf-5c78-4faa-8d45-e9549b82cf38 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:22.159322 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e95bf-5c78-4faa-8d45-e9549b82cf38 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:22.160795 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 618/1247] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:29:22.166762 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d49878-e17d-4df1-b637-6ad60e16b0ae None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:22.167396 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d49878-e17d-4df1-b637-6ad60e16b0ae None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:22.167396 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d49878-e17d-4df1-b637-6ad60e16b0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:22.173114 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:22.187639 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-36d49878-e17d-4df1-b637-6ad60e16b0ae None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:29:22.189014 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-36d49878-e17d-4df1-b637-6ad60e16b0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:29:22.290801 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 630/1248] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:29:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:29:22.315894 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e95bf-5c78-4faa-8d45-e9549b82cf38 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:22.342501 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:22.455319 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e95bf-5c78-4faa-8d45-e9549b82cf38 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:22.504438 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916cd94a-be2a-4b94-957d-c9401fc00578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:22.506332 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cd94a-be2a-4b94-957d-c9401fc00578 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:22.506836 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cd94a-be2a-4b94-957d-c9401fc00578 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:22.507249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cd94a-be2a-4b94-957d-c9401fc00578 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:22.522380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916cd94a-be2a-4b94-957d-c9401fc00578 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:29:22.530476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-916cd94a-be2a-4b94-957d-c9401fc00578 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:29:22.530476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:22.530476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:22.530476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:22.530476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:22.530476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:22.530476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:22.530476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:22.530476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:22.530476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:22.530476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:22.530476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:22.530476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:29:22.530476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:29:22.530476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:29:22.530476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:29:22.531571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:29:22.531571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:29:22.531571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:29:22.531571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:29:22.531571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:22.531571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:22.531571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:22.531571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:22.531571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:22.531571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:22.531571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:22.531571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:22.531571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:22.531571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:22.531571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:22.531571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:22.532448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:22.532448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:22.532448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:22.532448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:22.532448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:29:22.532448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:29:22.532448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:29:22.532448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:29:22.532448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:29:22.532448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:29:22.532448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:29:22.532448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:29:22.532448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 619/1249] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:29:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:22.538333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d49878-e17d-4df1-b637-6ad60e16b0ae None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:22.661454 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d49878-e17d-4df1-b637-6ad60e16b0ae None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:22.699262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:22.705417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8ba52-d66c-4515-860f-8386a7b0c23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:22.705962 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8ba52-d66c-4515-860f-8386a7b0c23c None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:22.706276 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8ba52-d66c-4515-860f-8386a7b0c23c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:22.706513 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8ba52-d66c-4515-860f-8386a7b0c23c None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:22.707276 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8ba52-d66c-4515-860f-8386a7b0c23c None admin] RBAC: Adding query filter params (name=admin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:22.707568 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8ba52-d66c-4515-860f-8386a7b0c23c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:22.708944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8ba52-d66c-4515-860f-8386a7b0c23c None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:22.717918 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 631/1250] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:29:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:22.723976 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916cd94a-be2a-4b94-957d-c9401fc00578 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:22.817047 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:22.848355 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916cd94a-be2a-4b94-957d-c9401fc00578 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:22.887335 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8eed2-f4b8-4add-b06d-e6b4b7d7efe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:22.888106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8eed2-f4b8-4add-b06d-e6b4b7d7efe0 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:22.888438 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8eed2-f4b8-4add-b06d-e6b4b7d7efe0 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:22.888756 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8eed2-f4b8-4add-b06d-e6b4b7d7efe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:22.902866 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8eed2-f4b8-4add-b06d-e6b4b7d7efe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:22.902866 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8eed2-f4b8-4add-b06d-e6b4b7d7efe0 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:22.924132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-cab8eed2-f4b8-4add-b06d-e6b4b7d7efe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:22.924132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:22.924132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:22.924132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:22.924132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:22.924132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:22.924132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:22.924132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:22.924132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:22.924132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:22.924132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:22.924132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:22.924132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:29:22.924132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:29:22.924132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:29:22.925264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:29:22.925264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:22.925264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:22.925264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:22.925264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:22.925264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:22.925264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:22.925264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:22.925264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:22.925264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:22.925264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:22.925264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:22.925264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:22.925264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:22.925264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:22.925264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:22.925264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:22.926176 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:29:22.926176 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:29:22.926176 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:22.926176 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:29:22.926176 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 620/1251] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:22.931779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8ba52-d66c-4515-860f-8386a7b0c23c None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:23.069119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8ba52-d66c-4515-860f-8386a7b0c23c None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:23.112225 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39357811-fe9f-486e-be21-7a34cd96728e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:23.112895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39357811-fe9f-486e-be21-7a34cd96728e None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:23.114206 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39357811-fe9f-486e-be21-7a34cd96728e None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:23.115665 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39357811-fe9f-486e-be21-7a34cd96728e None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:23.115847 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39357811-fe9f-486e-be21-7a34cd96728e None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:23.116172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39357811-fe9f-486e-be21-7a34cd96728e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:23.117804 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39357811-fe9f-486e-be21-7a34cd96728e None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:23.131750 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 632/1252] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:23.137338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8eed2-f4b8-4add-b06d-e6b4b7d7efe0 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:23.264993 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8eed2-f4b8-4add-b06d-e6b4b7d7efe0 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:23.299748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:23.310102 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fed935-f788-4a3a-80ef-1f0e1bdef44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:23.310979 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fed935-f788-4a3a-80ef-1f0e1bdef44f None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:23.311259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fed935-f788-4a3a-80ef-1f0e1bdef44f None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:23.311569 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fed935-f788-4a3a-80ef-1f0e1bdef44f None admin] PATH_INFO: `/v3/users/placement` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:23.319562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fed935-f788-4a3a-80ef-1f0e1bdef44f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:23.320942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fed935-f788-4a3a-80ef-1f0e1bdef44f None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:23.337992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-a6fed935-f788-4a3a-80ef-1f0e1bdef44f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:29:23.337992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:23.337992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:23.337992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:23.337992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:23.337992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:23.337992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:23.337992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:23.337992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:23.337992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:23.337992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:23.337992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:23.337992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:29:23.337992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:29:23.337992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:29:23.337992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:29:23.339308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:23.339308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:23.339308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:29:23.339308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:29:23.339308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:29:23.339308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:29:23.339308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:23.339308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:23.339308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:23.339308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:23.339308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:23.339308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:23.339308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:23.339308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:23.339308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:23.339308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:23.339308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:23.339308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:23.340334 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:23.340334 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:23.340334 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:29:23.340334 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:29:23.340334 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:29:23.340334 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:29:23.340334 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:29:23.340334 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:29:23.340334 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:29:23.340334 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:29:23.340334 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 621/1253] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:23.346091 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39357811-fe9f-486e-be21-7a34cd96728e None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:23.416607 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:23.468085 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39357811-fe9f-486e-be21-7a34cd96728e None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:23.508118 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db118673-11f2-4320-9a01-f5bb09adbfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:23.508793 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db118673-11f2-4320-9a01-f5bb09adbfd7 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:23.509153 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db118673-11f2-4320-9a01-f5bb09adbfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:23.509503 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db118673-11f2-4320-9a01-f5bb09adbfd7 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:23.510562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db118673-11f2-4320-9a01-f5bb09adbfd7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:23.510934 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db118673-11f2-4320-9a01-f5bb09adbfd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:23.512547 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db118673-11f2-4320-9a01-f5bb09adbfd7 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:23.532430 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 633/1254] 10.222.0.71 () {56 vars in 1143 bytes} [Thu Jul 21 20:29:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:23.538944 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fed935-f788-4a3a-80ef-1f0e1bdef44f None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:23.657098 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fed935-f788-4a3a-80ef-1f0e1bdef44f None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:23.701511 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531001f-781c-410a-a5c3-c8de922a464a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:23.702189 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531001f-781c-410a-a5c3-c8de922a464a None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:23.702476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531001f-781c-410a-a5c3-c8de922a464a None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:23.702756 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531001f-781c-410a-a5c3-c8de922a464a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:23.774823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:23.790456 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531001f-781c-410a-a5c3-c8de922a464a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:23.792117 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531001f-781c-410a-a5c3-c8de922a464a None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:23.964111 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 622/1255] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:29:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:29:23.972263 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db118673-11f2-4320-9a01-f5bb09adbfd7 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:24.111639 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db118673-11f2-4320-9a01-f5bb09adbfd7 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:24.128378 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:24.160153 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b71c8-93a9-4747-b3e4-8c7cec33f049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:24.161479 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b71c8-93a9-4747-b3e4-8c7cec33f049 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:24.161932 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b71c8-93a9-4747-b3e4-8c7cec33f049 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:24.162508 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b71c8-93a9-4747-b3e4-8c7cec33f049 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:24.182229 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b71c8-93a9-4747-b3e4-8c7cec33f049 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:24.184269 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b71c8-93a9-4747-b3e4-8c7cec33f049 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:24.207004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-920b71c8-93a9-4747-b3e4-8c7cec33f049 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:24.207004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:24.207004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:24.207004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:24.207004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:24.207004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:24.207004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:24.207004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:24.207004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:24.207004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:24.207004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:24.207004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:24.207004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:29:24.207004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:29:24.207004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:29:24.208240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:29:24.208240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:24.208240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:24.208240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:24.208240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:24.208240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:24.208240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:24.208240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:24.208240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:24.208240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:24.208240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:24.208240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:24.208240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:24.208240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:24.208240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:24.208240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:24.208240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:24.210605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:29:24.210605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:29:24.210605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:24.210605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:24.210605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 634/1256] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:24.215637 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531001f-781c-410a-a5c3-c8de922a464a None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:24.334056 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:24.349942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531001f-781c-410a-a5c3-c8de922a464a None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:24.399262 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06b61b-7efd-4d87-8149-747f30fea407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:24.399973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06b61b-7efd-4d87-8149-747f30fea407 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:24.400277 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06b61b-7efd-4d87-8149-747f30fea407 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:24.400547 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06b61b-7efd-4d87-8149-747f30fea407 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:24.401370 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06b61b-7efd-4d87-8149-747f30fea407 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:24.401673 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06b61b-7efd-4d87-8149-747f30fea407 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:24.403361 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06b61b-7efd-4d87-8149-747f30fea407 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:24.417665 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 623/1257] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:24.424644 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b71c8-93a9-4747-b3e4-8c7cec33f049 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:24.555104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b71c8-93a9-4747-b3e4-8c7cec33f049 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:24.596042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9e03e-eef2-47d1-82ff-1a53de9ab6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:24.596849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9e03e-eef2-47d1-82ff-1a53de9ab6dd None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:24.597154 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9e03e-eef2-47d1-82ff-1a53de9ab6dd None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:24.597431 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9e03e-eef2-47d1-82ff-1a53de9ab6dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:24.605571 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9e03e-eef2-47d1-82ff-1a53de9ab6dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:24.607046 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9e03e-eef2-47d1-82ff-1a53de9ab6dd None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:24.624500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-47a9e03e-eef2-47d1-82ff-1a53de9ab6dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:29:24.624500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:24.624500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:24.624500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:24.624500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:24.624500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:24.624500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:24.624500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:24.624500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:24.624500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:24.624500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:24.624500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:24.624500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:29:24.624500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:29:24.624500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:29:24.625897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:29:24.625897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:24.625897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:24.625897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:24.625897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:24.625897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:24.625897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:24.625897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:24.625897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:24.625897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:24.625897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:24.625897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:24.625897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:24.625897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:24.625897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:24.625897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:24.625897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:24.626895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:29:24.626895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:29:24.626895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:29:24.626895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:29:24.626895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:29:24.626895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:29:24.626895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:29:24.626895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:24.626895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 635/1258] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:29:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:24.632476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06b61b-7efd-4d87-8149-747f30fea407 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:24.752365 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06b61b-7efd-4d87-8149-747f30fea407 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:24.791714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37511dac-af85-4b36-8c19-25a1ea8b0bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:24.792376 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37511dac-af85-4b36-8c19-25a1ea8b0bb5 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:24.792758 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37511dac-af85-4b36-8c19-25a1ea8b0bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:24.793041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37511dac-af85-4b36-8c19-25a1ea8b0bb5 None admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:24.793860 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37511dac-af85-4b36-8c19-25a1ea8b0bb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:24.794272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37511dac-af85-4b36-8c19-25a1ea8b0bb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:24.795855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37511dac-af85-4b36-8c19-25a1ea8b0bb5 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:24.809786 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 624/1259] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:29:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:24.816226 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9e03e-eef2-47d1-82ff-1a53de9ab6dd None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:24.844600 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:24.924313 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9e03e-eef2-47d1-82ff-1a53de9ab6dd None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:24.964789 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca2c53-ed22-4616-a59e-64ae3807e0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:24.965513 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca2c53-ed22-4616-a59e-64ae3807e0fc None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:24.965791 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca2c53-ed22-4616-a59e-64ae3807e0fc None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:24.966127 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca2c53-ed22-4616-a59e-64ae3807e0fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:24.966968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca2c53-ed22-4616-a59e-64ae3807e0fc None admin] RBAC: Adding query filter params (role.id=19f5e72c0be44a1197c7288de82e79e8, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=364e80ec79b94812a1df76c62c2b13a0) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:24.967279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca2c53-ed22-4616-a59e-64ae3807e0fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:24.968715 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca2c53-ed22-4616-a59e-64ae3807e0fc None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:24.975592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 636/1260] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:29:24 2022] GET /identity/v3/role_assignments?user.id=364e80ec79b94812a1df76c62c2b13a0&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:27.037934 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b22123-e752-421e-890e-9379e800bb8c None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:27.038476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b22123-e752-421e-890e-9379e800bb8c None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:27.038476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b22123-e752-421e-890e-9379e800bb8c None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:27.039921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 625/1261] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:29:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:29:27.045048 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d19c11-07b1-44fa-adb1-1292a7e972bb None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:27.045520 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d19c11-07b1-44fa-adb1-1292a7e972bb None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:27.045623 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d19c11-07b1-44fa-adb1-1292a7e972bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:27.065247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-e3d19c11-07b1-44fa-adb1-1292a7e972bb None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:29:27.066470 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-e3d19c11-07b1-44fa-adb1-1292a7e972bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:29:27.135682 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 637/1262] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:29:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:29:27.159450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b22123-e752-421e-890e-9379e800bb8c None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:27.171094 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:27.264573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b22123-e752-421e-890e-9379e800bb8c None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:27.297741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde263f6-b420-4dc8-a8a8-be9bd963c37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:27.298452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde263f6-b420-4dc8-a8a8-be9bd963c37c None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:27.298735 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde263f6-b420-4dc8-a8a8-be9bd963c37c None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:27.299010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde263f6-b420-4dc8-a8a8-be9bd963c37c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:27.309564 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde263f6-b420-4dc8-a8a8-be9bd963c37c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:29:27.317178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-bde263f6-b420-4dc8-a8a8-be9bd963c37c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:29:27.317178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:27.317178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:27.317178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:27.317178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:27.317178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:27.317178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:27.317178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:27.317178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:27.317178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:27.317178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:27.317178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:27.317178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:29:27.317178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:29:27.317178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:29:27.317178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:29:27.318380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:29:27.318380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:29:27.318380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:29:27.318380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:29:27.318380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:27.318380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:27.318380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:27.318380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:27.318380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:27.318380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:27.318380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:27.318380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:27.318380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:27.318380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:27.318380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:27.318380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:27.320021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:27.320021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:27.320021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:27.320021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:27.320021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:29:27.320021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:29:27.320021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:29:27.320021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:29:27.320021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:29:27.320021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:29:27.320021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:29:27.320021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:29:27.320021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 626/1263] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:29:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:27.325094 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d19c11-07b1-44fa-adb1-1292a7e972bb None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:27.343241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:27.423237 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d19c11-07b1-44fa-adb1-1292a7e972bb None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:27.459292 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bb86b-ca9d-4d92-8834-c1dbab20a7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:27.459964 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bb86b-ca9d-4d92-8834-c1dbab20a7c7 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:27.460253 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bb86b-ca9d-4d92-8834-c1dbab20a7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:27.460528 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bb86b-ca9d-4d92-8834-c1dbab20a7c7 None admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:27.461306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bb86b-ca9d-4d92-8834-c1dbab20a7c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:27.461616 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bb86b-ca9d-4d92-8834-c1dbab20a7c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:27.462980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bb86b-ca9d-4d92-8834-c1dbab20a7c7 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:27.471386 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 638/1264] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:29:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:27.478026 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde263f6-b420-4dc8-a8a8-be9bd963c37c None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:27.544894 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:27.579444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde263f6-b420-4dc8-a8a8-be9bd963c37c None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:27.613132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46012bb6-e02c-47ec-abe6-8bda47ebb90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:27.613921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46012bb6-e02c-47ec-abe6-8bda47ebb90b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:27.614839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46012bb6-e02c-47ec-abe6-8bda47ebb90b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:27.615047 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46012bb6-e02c-47ec-abe6-8bda47ebb90b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:27.625830 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46012bb6-e02c-47ec-abe6-8bda47ebb90b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:27.627284 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46012bb6-e02c-47ec-abe6-8bda47ebb90b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:27.644505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-46012bb6-e02c-47ec-abe6-8bda47ebb90b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:27.644505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:27.644505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:27.644505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:27.644505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:27.644505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:27.644505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:27.644505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:27.644505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:27.644505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:27.644505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:27.644505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:27.644505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:29:27.644505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:29:27.644505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:29:27.645568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:29:27.645568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:27.645568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:27.645568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:27.645568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:27.645568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:27.645568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:27.645568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:27.645568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:27.645568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:27.645568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:27.645568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:27.645568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:27.645568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:27.645568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:27.645568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:27.645568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:27.646663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:29:27.646663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:29:27.646663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:27.646663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:29:27.646663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 627/1265] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:27.652253 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bb86b-ca9d-4d92-8834-c1dbab20a7c7 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:27.737674 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:27.758122 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bb86b-ca9d-4d92-8834-c1dbab20a7c7 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:27.794001 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11733cd8-54fd-416a-954e-cb3a8f0a5ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:27.794697 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11733cd8-54fd-416a-954e-cb3a8f0a5ba0 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:27.794979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11733cd8-54fd-416a-954e-cb3a8f0a5ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:27.795255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11733cd8-54fd-416a-954e-cb3a8f0a5ba0 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:27.795985 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11733cd8-54fd-416a-954e-cb3a8f0a5ba0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:27.796292 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11733cd8-54fd-416a-954e-cb3a8f0a5ba0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:27.797700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11733cd8-54fd-416a-954e-cb3a8f0a5ba0 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:27.811097 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 639/1266] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:27.818037 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46012bb6-e02c-47ec-abe6-8bda47ebb90b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:27.922594 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46012bb6-e02c-47ec-abe6-8bda47ebb90b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:27.942049 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:27.956713 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934f475-992f-4f2e-b0c3-ff76b5b85f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:27.957568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934f475-992f-4f2e-b0c3-ff76b5b85f7b None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:27.957852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934f475-992f-4f2e-b0c3-ff76b5b85f7b None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:27.958664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934f475-992f-4f2e-b0c3-ff76b5b85f7b None admin] PATH_INFO: `/v3/users/placement` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:27.967082 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934f475-992f-4f2e-b0c3-ff76b5b85f7b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:27.968558 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934f475-992f-4f2e-b0c3-ff76b5b85f7b None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:27.983982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-0934f475-992f-4f2e-b0c3-ff76b5b85f7b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:29:27.983982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:27.983982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:27.983982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:27.983982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:27.983982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:27.983982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:27.983982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:27.983982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:27.983982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:27.983982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:27.983982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:27.983982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:29:27.983982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:29:27.983982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:29:27.983982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:29:27.985235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:27.985235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:27.985235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:29:27.985235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:29:27.985235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:29:27.985235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:29:27.985235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:27.985235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:27.985235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:27.985235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:27.985235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:27.985235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:27.985235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:27.985235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:27.985235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:27.985235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:27.985235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:27.985235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:27.986259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:27.986259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:27.986259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:29:27.986259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:29:27.986259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:29:27.986259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:29:27.986259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:29:27.986259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:29:27.986259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:29:27.986259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:29:27.986259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 628/1267] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:27.991281 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11733cd8-54fd-416a-954e-cb3a8f0a5ba0 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:28.101992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11733cd8-54fd-416a-954e-cb3a8f0a5ba0 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:28.136231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b96b1-9c3d-4d71-acde-b41bc2f5d484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:28.136902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b96b1-9c3d-4d71-acde-b41bc2f5d484 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:28.137235 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b96b1-9c3d-4d71-acde-b41bc2f5d484 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:28.137513 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b96b1-9c3d-4d71-acde-b41bc2f5d484 None admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:28.138352 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b96b1-9c3d-4d71-acde-b41bc2f5d484 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:28.138663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b96b1-9c3d-4d71-acde-b41bc2f5d484 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:28.140087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b96b1-9c3d-4d71-acde-b41bc2f5d484 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:28.149616 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:28.158378 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 640/1268] 10.222.0.71 () {56 vars in 1143 bytes} [Thu Jul 21 20:29:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:28.164753 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934f475-992f-4f2e-b0c3-ff76b5b85f7b None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:28.267741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934f475-992f-4f2e-b0c3-ff76b5b85f7b None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:28.302101 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46852043-3d9c-4dc6-9d87-f1cbb3591c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:28.302895 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46852043-3d9c-4dc6-9d87-f1cbb3591c87 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:28.303176 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46852043-3d9c-4dc6-9d87-f1cbb3591c87 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:28.303451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46852043-3d9c-4dc6-9d87-f1cbb3591c87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:28.314879 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46852043-3d9c-4dc6-9d87-f1cbb3591c87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:28.316347 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46852043-3d9c-4dc6-9d87-f1cbb3591c87 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:28.334777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-46852043-3d9c-4dc6-9d87-f1cbb3591c87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:28.334777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:28.334777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:28.334777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:28.334777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:28.334777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:28.334777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:28.334777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:28.334777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:28.334777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:28.334777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:28.334777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:28.334777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:29:28.334777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:29:28.334777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:29:28.336084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:29:28.336084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:28.336084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:28.336084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:28.336084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:28.336084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:28.336084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:28.336084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:28.336084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:28.336084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:28.336084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:28.336084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:28.336084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:28.336084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:28.336084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:28.336084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:28.336084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:28.337290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:29:28.337290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:29:28.337290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:28.337290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:29:28.337290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 629/1269] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:28.341975 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b96b1-9c3d-4d71-acde-b41bc2f5d484 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:28.442817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b96b1-9c3d-4d71-acde-b41bc2f5d484 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:28.480625 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44e53b-f636-402b-8484-386f7e955d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:28.481309 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44e53b-f636-402b-8484-386f7e955d9b None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:28.481590 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44e53b-f636-402b-8484-386f7e955d9b None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:28.481862 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44e53b-f636-402b-8484-386f7e955d9b None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:28.482593 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44e53b-f636-402b-8484-386f7e955d9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:28.482903 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44e53b-f636-402b-8484-386f7e955d9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:28.484293 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44e53b-f636-402b-8484-386f7e955d9b None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:28.496378 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 641/1270] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:28.502377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46852043-3d9c-4dc6-9d87-f1cbb3591c87 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:28.537700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:28.607435 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46852043-3d9c-4dc6-9d87-f1cbb3591c87 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:28.642356 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309042dc-905d-48d9-bb40-fdc9deec7a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:28.643162 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309042dc-905d-48d9-bb40-fdc9deec7a31 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:28.643447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309042dc-905d-48d9-bb40-fdc9deec7a31 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:28.643720 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309042dc-905d-48d9-bb40-fdc9deec7a31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:28.650956 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309042dc-905d-48d9-bb40-fdc9deec7a31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:28.652416 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309042dc-905d-48d9-bb40-fdc9deec7a31 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:28.667552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-309042dc-905d-48d9-bb40-fdc9deec7a31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:29:28.667552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:28.667552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:28.667552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:28.667552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:28.667552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:28.667552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:28.667552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:28.667552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:28.667552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:28.667552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:28.667552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:28.667552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:29:28.667552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:29:28.667552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:29:28.669340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:29:28.669340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:28.669340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:28.669340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:28.669340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:28.669340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:28.669340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:28.669340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:28.669340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:28.669340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:28.669340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:28.669340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:28.669340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:28.669340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:28.669340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:28.669340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:28.669340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:28.671242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:29:28.671242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:29:28.671242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:29:28.671242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:29:28.671242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:29:28.671242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:29:28.671242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:29:28.671242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:29:28.671242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 630/1271] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:29:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:28.674762 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44e53b-f636-402b-8484-386f7e955d9b None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:28.734536 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:28.777176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44e53b-f636-402b-8484-386f7e955d9b None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:28.811918 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f4cbd-6b10-4d15-951e-b9fe0428dfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:28.812482 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f4cbd-6b10-4d15-951e-b9fe0428dfe5 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:28.812758 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f4cbd-6b10-4d15-951e-b9fe0428dfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:28.813028 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f4cbd-6b10-4d15-951e-b9fe0428dfe5 None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:28.813842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f4cbd-6b10-4d15-951e-b9fe0428dfe5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:28.814148 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f4cbd-6b10-4d15-951e-b9fe0428dfe5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:28.815579 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f4cbd-6b10-4d15-951e-b9fe0428dfe5 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:28.828080 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 642/1272] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:29:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:28.834149 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309042dc-905d-48d9-bb40-fdc9deec7a31 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:28.925368 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:28.934370 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309042dc-905d-48d9-bb40-fdc9deec7a31 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:28.969464 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf4ed2-e918-4441-8c14-909e74638b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:28.971015 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf4ed2-e918-4441-8c14-909e74638b0c None admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:28.971015 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf4ed2-e918-4441-8c14-909e74638b0c None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:28.971789 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf4ed2-e918-4441-8c14-909e74638b0c None admin] PATH_INFO: `/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/364e80ec79b94812a1df76c62c2b13a0/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:28.973611 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf4ed2-e918-4441-8c14-909e74638b0c None admin] RBAC: Authorizing `identity:create_grant(project_id=37e4afa595894a9994a8e6981ebb7c41, user_id=364e80ec79b94812a1df76c62c2b13a0, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:28.975070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf4ed2-e918-4441-8c14-909e74638b0c None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:28.992194 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 631/1273] 10.222.0.71 () {58 vars in 1439 bytes} [Thu Jul 21 20:29:28 2022] PUT /identity/v3/projects/37e4afa595894a9994a8e6981ebb7c41/users/364e80ec79b94812a1df76c62c2b13a0/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:29:31.077258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacd0c1-2702-41fe-9fe5-64f50a5f06c1 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:31.077833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacd0c1-2702-41fe-9fe5-64f50a5f06c1 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:31.077833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacd0c1-2702-41fe-9fe5-64f50a5f06c1 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:31.079182 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 643/1274] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:29:31.085308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f37fd-4f28-4d33-a917-61bb34f57c7d None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:31.085711 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f37fd-4f28-4d33-a917-61bb34f57c7d None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:31.085861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f37fd-4f28-4d33-a917-61bb34f57c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:31.102982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-1d5f37fd-4f28-4d33-a917-61bb34f57c7d None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:29:31.104175 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-1d5f37fd-4f28-4d33-a917-61bb34f57c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:29:31.169401 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 632/1275] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:29:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:29:31.195077 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cacd0c1-2702-41fe-9fe5-64f50a5f06c1 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:31.293999 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cacd0c1-2702-41fe-9fe5-64f50a5f06c1 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:31.310627 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:31.327833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2653e-f5cf-4c79-a6c8-8946df1958f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:31.328547 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2653e-f5cf-4c79-a6c8-8946df1958f6 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:31.328831 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2653e-f5cf-4c79-a6c8-8946df1958f6 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:31.329134 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2653e-f5cf-4c79-a6c8-8946df1958f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:31.339828 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2653e-f5cf-4c79-a6c8-8946df1958f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:29:31.348087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-e2b2653e-f5cf-4c79-a6c8-8946df1958f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:29:31.348087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:31.348087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:31.348087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:31.348087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:31.348087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:31.348087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:31.348087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:31.348087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:31.348087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:31.348087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:31.348087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:31.348087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:29:31.348087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:29:31.348087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:29:31.348087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:29:31.349275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:29:31.349275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:29:31.349275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:29:31.349275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:29:31.349275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:31.349275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:31.349275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:31.349275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:31.349275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:31.349275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:31.349275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:31.349275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:31.349275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:31.349275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:31.349275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:31.349275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:31.350209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:31.350209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:31.350209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:31.350209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:31.350209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:29:31.350209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:29:31.350209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:29:31.350209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:29:31.350209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:29:31.350209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:29:31.350209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:29:31.350209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:31.350209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 644/1276] 10.222.0.71 () {56 vars in 1096 bytes} [Thu Jul 21 20:29:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:31.355511 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f37fd-4f28-4d33-a917-61bb34f57c7d None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:31.438469 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:31.459099 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f37fd-4f28-4d33-a917-61bb34f57c7d None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:31.493792 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06390ef5-4578-4a99-9ae2-967f0a3ab2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:31.494450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06390ef5-4578-4a99-9ae2-967f0a3ab2fb None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:31.494728 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06390ef5-4578-4a99-9ae2-967f0a3ab2fb None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:31.495034 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06390ef5-4578-4a99-9ae2-967f0a3ab2fb None admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:31.495772 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06390ef5-4578-4a99-9ae2-967f0a3ab2fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:31.496077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06390ef5-4578-4a99-9ae2-967f0a3ab2fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:31.497476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06390ef5-4578-4a99-9ae2-967f0a3ab2fb None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:31.507502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 633/1277] 10.222.0.71 () {56 vars in 1099 bytes} [Thu Jul 21 20:29:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:31.513376 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2653e-f5cf-4c79-a6c8-8946df1958f6 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:31.611347 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2653e-f5cf-4c79-a6c8-8946df1958f6 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:31.644121 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43773d-6c50-491f-bce8-64a952547f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:31.644890 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43773d-6c50-491f-bce8-64a952547f30 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:31.645197 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43773d-6c50-491f-bce8-64a952547f30 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:31.645988 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43773d-6c50-491f-bce8-64a952547f30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:31.656089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43773d-6c50-491f-bce8-64a952547f30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:31.657563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43773d-6c50-491f-bce8-64a952547f30 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:31.674603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-ad43773d-6c50-491f-bce8-64a952547f30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:31.674603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:31.674603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:31.674603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:31.674603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:31.674603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:31.674603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:31.674603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:31.674603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:31.674603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:31.674603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:31.674603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:31.674603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:29:31.674603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:29:31.674603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:29:31.675706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:29:31.675706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:31.675706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:31.675706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:31.675706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:31.675706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:31.675706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:31.675706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:31.675706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:31.675706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:31.675706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:31.675706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:31.675706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:31.675706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:31.675706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:31.675706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:31.675706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:31.676781 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:29:31.676781 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:29:31.676781 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:31.676781 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:31.676781 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 645/1278] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:31.681734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06390ef5-4578-4a99-9ae2-967f0a3ab2fb None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:31.779434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06390ef5-4578-4a99-9ae2-967f0a3ab2fb None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:31.812102 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c87f52-5840-40db-b267-7e702931b281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:31.812774 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c87f52-5840-40db-b267-7e702931b281 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:31.813040 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c87f52-5840-40db-b267-7e702931b281 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:31.813375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c87f52-5840-40db-b267-7e702931b281 None admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:31.814069 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c87f52-5840-40db-b267-7e702931b281 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:31.814371 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c87f52-5840-40db-b267-7e702931b281 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:31.815729 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c87f52-5840-40db-b267-7e702931b281 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:31.828072 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 634/1279] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:31.834295 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43773d-6c50-491f-bce8-64a952547f30 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:31.865368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:31.933491 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43773d-6c50-491f-bce8-64a952547f30 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:31.967848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac46321-466a-4b24-8162-baa177aa39ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:31.968705 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac46321-466a-4b24-8162-baa177aa39ca None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:31.968926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac46321-466a-4b24-8162-baa177aa39ca None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:31.969233 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac46321-466a-4b24-8162-baa177aa39ca None admin] PATH_INFO: `/v3/users/placement` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:31.976966 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac46321-466a-4b24-8162-baa177aa39ca None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:31.978516 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac46321-466a-4b24-8162-baa177aa39ca None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:31.992548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-1ac46321-466a-4b24-8162-baa177aa39ca None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:29:31.992548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:31.992548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:31.992548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:31.992548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:31.992548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:31.992548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:31.992548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:31.992548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:31.992548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:31.992548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:31.992548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:31.992548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:29:31.992548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:29:31.992548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:29:31.992548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:29:31.993724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:31.993724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:31.993724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:29:31.993724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:29:31.993724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:29:31.993724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:29:31.993724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:31.993724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:31.993724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:31.993724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:31.993724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:31.993724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:31.993724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:31.993724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:31.993724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:31.993724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:31.993724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:31.993724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:31.994888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:31.994888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:31.994888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:29:31.994888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:29:31.994888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:29:31.994888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:29:31.994888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:29:31.994888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:29:31.994888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:29:31.994888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:31.994888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 646/1280] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:29:31.999576 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c87f52-5840-40db-b267-7e702931b281 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:32.003309 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:32.122837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c87f52-5840-40db-b267-7e702931b281 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:32.156617 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da95c09-9ebf-49b5-9877-accbfa4b576c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:32.157305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da95c09-9ebf-49b5-9877-accbfa4b576c None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:32.157591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da95c09-9ebf-49b5-9877-accbfa4b576c None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:32.157867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da95c09-9ebf-49b5-9877-accbfa4b576c None admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:32.158703 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da95c09-9ebf-49b5-9877-accbfa4b576c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:32.159153 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da95c09-9ebf-49b5-9877-accbfa4b576c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:32.160583 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da95c09-9ebf-49b5-9877-accbfa4b576c None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:32.177196 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 635/1281] 10.222.0.71 () {56 vars in 1143 bytes} [Thu Jul 21 20:29:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:32.182908 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac46321-466a-4b24-8162-baa177aa39ca None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:32.272064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:32.281639 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac46321-466a-4b24-8162-baa177aa39ca None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:32.314184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d9949-04f5-4595-9646-b9aa9dd6b8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:32.314821 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d9949-04f5-4595-9646-b9aa9dd6b8f1 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:32.315098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d9949-04f5-4595-9646-b9aa9dd6b8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:32.315369 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d9949-04f5-4595-9646-b9aa9dd6b8f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:32.382868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d9949-04f5-4595-9646-b9aa9dd6b8f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:32.384224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d9949-04f5-4595-9646-b9aa9dd6b8f1 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:32.469542 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:32.480911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 647/1282] 10.222.0.71 () {58 vars in 1282 bytes} [Thu Jul 21 20:29:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:29:32.486691 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da95c09-9ebf-49b5-9877-accbfa4b576c None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:32.546344 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:32.583973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da95c09-9ebf-49b5-9877-accbfa4b576c None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:32.617161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37088a53-ff1a-4ed5-823f-340d023179bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:32.617939 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37088a53-ff1a-4ed5-823f-340d023179bc None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:32.618222 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37088a53-ff1a-4ed5-823f-340d023179bc None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:32.618495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37088a53-ff1a-4ed5-823f-340d023179bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:32.628285 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37088a53-ff1a-4ed5-823f-340d023179bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:32.629800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37088a53-ff1a-4ed5-823f-340d023179bc None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:32.647350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-37088a53-ff1a-4ed5-823f-340d023179bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:32.647350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:32.647350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:32.647350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:32.647350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:32.647350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:32.647350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:32.647350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:32.647350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:32.647350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:32.647350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:32.647350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:32.647350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:29:32.647350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:29:32.647350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:29:32.648406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:29:32.648406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:32.648406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:32.648406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:32.648406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:32.648406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:32.648406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:32.648406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:32.648406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:32.648406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:32.648406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:32.648406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:32.648406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:32.648406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:32.648406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:32.648406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:32.648406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:32.649544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:29:32.649544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:29:32.649544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:29:32.649544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:29:32.649544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 636/1283] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:29:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:32.654397 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d9949-04f5-4595-9646-b9aa9dd6b8f1 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:32.752021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d9949-04f5-4595-9646-b9aa9dd6b8f1 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:32.786360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f42977-90b6-43d6-bb31-f4bce5c48519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:32.787011 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f42977-90b6-43d6-bb31-f4bce5c48519 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:32.787293 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f42977-90b6-43d6-bb31-f4bce5c48519 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:32.787569 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f42977-90b6-43d6-bb31-f4bce5c48519 None admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:32.788299 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f42977-90b6-43d6-bb31-f4bce5c48519 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:32.788606 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f42977-90b6-43d6-bb31-f4bce5c48519 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:32.790019 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f42977-90b6-43d6-bb31-f4bce5c48519 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:32.800853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 648/1284] 10.222.0.71 () {56 vars in 1109 bytes} [Thu Jul 21 20:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:32.806521 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37088a53-ff1a-4ed5-823f-340d023179bc None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:32.904985 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37088a53-ff1a-4ed5-823f-340d023179bc None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:32.919451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:32.939422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe4d6a-e978-4c88-bfb2-4cb9d82e80a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:32.940202 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe4d6a-e978-4c88-bfb2-4cb9d82e80a5 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:32.940484 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe4d6a-e978-4c88-bfb2-4cb9d82e80a5 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:32.940758 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe4d6a-e978-4c88-bfb2-4cb9d82e80a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:32.947871 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe4d6a-e978-4c88-bfb2-4cb9d82e80a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:32.949356 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe4d6a-e978-4c88-bfb2-4cb9d82e80a5 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:32.963664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application [None req-eebe4d6a-e978-4c88-bfb2-4cb9d82e80a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:29:32.963664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:32.963664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:32.963664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:32.963664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:32.963664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:32.963664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:32.963664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:32.963664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:32.963664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:32.963664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:32.963664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:32.963664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:29:32.963664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:29:32.963664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:29:32.964800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:29:32.964800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:32.964800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:32.964800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:32.964800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:32.964800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:32.964800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:32.964800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:32.964800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:32.964800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:32.964800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:32.964800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:32.964800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:32.964800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:32.964800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:32.964800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:32.964800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:32.965874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:29:32.965874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:29:32.965874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:29:32.965874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:29:32.965874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:29:32.965874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:29:32.965874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:29:32.965874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: ERROR keystone.server.flask.application Jul 21 20:29:32.965874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 637/1285] 10.222.0.71 () {56 vars in 1111 bytes} [Thu Jul 21 20:29:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:32.970998 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f42977-90b6-43d6-bb31-f4bce5c48519 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:33.023175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:33.072719 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f42977-90b6-43d6-bb31-f4bce5c48519 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:33.106603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86af4b8-6915-48d3-a506-4bc21e511149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:33.107247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86af4b8-6915-48d3-a506-4bc21e511149 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:33.107523 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86af4b8-6915-48d3-a506-4bc21e511149 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:33.107792 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86af4b8-6915-48d3-a506-4bc21e511149 None admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:33.108574 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86af4b8-6915-48d3-a506-4bc21e511149 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:33.108879 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86af4b8-6915-48d3-a506-4bc21e511149 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:33.110341 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86af4b8-6915-48d3-a506-4bc21e511149 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:33.121918 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 649/1286] 10.222.0.71 () {56 vars in 1148 bytes} [Thu Jul 21 20:29:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:33.128124 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe4d6a-e978-4c88-bfb2-4cb9d82e80a5 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:33.229273 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe4d6a-e978-4c88-bfb2-4cb9d82e80a5 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:33.261192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139d4b5-c157-4639-adab-2df382a4f7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:33.261843 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139d4b5-c157-4639-adab-2df382a4f7af None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:33.262123 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139d4b5-c157-4639-adab-2df382a4f7af None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:33.262398 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139d4b5-c157-4639-adab-2df382a4f7af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:33.263232 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139d4b5-c157-4639-adab-2df382a4f7af None admin] RBAC: Adding query filter params (role.id=19f5e72c0be44a1197c7288de82e79e8, scope.project.id=37e4afa595894a9994a8e6981ebb7c41, user.id=364e80ec79b94812a1df76c62c2b13a0) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:33.263545 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139d4b5-c157-4639-adab-2df382a4f7af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:33.265003 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139d4b5-c157-4639-adab-2df382a4f7af None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:33.271348 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 638/1287] 10.222.0.71 () {56 vars in 1374 bytes} [Thu Jul 21 20:29:33 2022] GET /identity/v3/role_assignments?user.id=364e80ec79b94812a1df76c62c2b13a0&scope.project.id=37e4afa595894a9994a8e6981ebb7c41&role.id=19f5e72c0be44a1197c7288de82e79e8 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:35.328413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd0f8f-2388-4b83-9535-f1a5185125ef None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:35.328982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd0f8f-2388-4b83-9535-f1a5185125ef None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:35.328982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd0f8f-2388-4b83-9535-f1a5185125ef None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:35.330382 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 650/1288] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:29:35.336134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79b3b9-8d7b-4740-89dd-d279671d133e None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:35.336553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79b3b9-8d7b-4740-89dd-d279671d133e None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:35.336687 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79b3b9-8d7b-4740-89dd-d279671d133e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:35.354283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-ef79b3b9-8d7b-4740-89dd-d279671d133e None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:29:35.355492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-ef79b3b9-8d7b-4740-89dd-d279671d133e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:29:35.384023 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:35.420806 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 639/1289] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:29:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:29:35.443495 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd0f8f-2388-4b83-9535-f1a5185125ef None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:35.545331 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd0f8f-2388-4b83-9535-f1a5185125ef None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:35.556554 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:35.579735 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d05f41-d7ed-44aa-a2b0-d00fcdb51ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:35.580455 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d05f41-d7ed-44aa-a2b0-d00fcdb51ff1 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:35.580743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d05f41-d7ed-44aa-a2b0-d00fcdb51ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:35.581017 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d05f41-d7ed-44aa-a2b0-d00fcdb51ff1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:35.581710 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d05f41-d7ed-44aa-a2b0-d00fcdb51ff1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:35.583610 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d05f41-d7ed-44aa-a2b0-d00fcdb51ff1 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:35.595530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-47d05f41-d7ed-44aa-a2b0-d00fcdb51ff1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 20:29:35.595530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:35.595530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:29:35.595530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:29:35.595530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:29:35.595530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:29:35.595530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:29:35.595530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:29:35.595530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 20:29:35.595530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:35.595530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:29:35.595530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:35.595530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:35.595530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:35.595530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:35.595530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:35.595530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:35.596798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:35.596798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:35.596798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:35.596798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:35.596798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:35.596798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:35.596798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:29:35.596798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:29:35.596798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:29:35.596798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:29:35.596798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:35.596798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:35.596798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:35.596798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:35.596798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:35.596798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:35.598079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:35.598079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:35.598079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:35.598079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:35.598079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:35.598079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:35.598079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:35.598079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:35.598079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:35.598079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:35.598079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:29:35.598079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:29:35.598079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 20:29:35.598079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:35.598079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 651/1290] 10.222.0.71 () {56 vars in 1117 bytes} [Thu Jul 21 20:29:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:35.603368 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79b3b9-8d7b-4740-89dd-d279671d133e None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:35.699140 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79b3b9-8d7b-4740-89dd-d279671d133e None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:35.732328 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16878e44-7547-4462-b041-fb1ad73d4a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:35.732984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16878e44-7547-4462-b041-fb1ad73d4a36 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:35.733296 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16878e44-7547-4462-b041-fb1ad73d4a36 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:35.733576 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16878e44-7547-4462-b041-fb1ad73d4a36 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:35.734282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16878e44-7547-4462-b041-fb1ad73d4a36 None admin] RBAC: Adding query filter params (name=placement) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:35.734591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16878e44-7547-4462-b041-fb1ad73d4a36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:35.735954 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16878e44-7547-4462-b041-fb1ad73d4a36 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:35.740724 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 640/1291] 10.222.0.71 () {56 vars in 1116 bytes} [Thu Jul 21 20:29:35 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:35.746465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d05f41-d7ed-44aa-a2b0-d00fcdb51ff1 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:35.845586 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d05f41-d7ed-44aa-a2b0-d00fcdb51ff1 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:35.879067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6ba42-7cf1-4486-a52b-0300dd65899d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:35.880258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6ba42-7cf1-4486-a52b-0300dd65899d None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:35.880538 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6ba42-7cf1-4486-a52b-0300dd65899d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:35.880813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6ba42-7cf1-4486-a52b-0300dd65899d None admin] PATH_INFO: `/v3/services` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:35.881551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6ba42-7cf1-4486-a52b-0300dd65899d None admin] RBAC: Adding query filter params (type=placement) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:35.882316 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6ba42-7cf1-4486-a52b-0300dd65899d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:35.883234 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6ba42-7cf1-4486-a52b-0300dd65899d None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:35.887679 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 652/1292] 10.222.0.71 () {56 vars in 1116 bytes} [Thu Jul 21 20:29:35 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:37.897163 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b2202-1f74-4786-9930-836e6c4cf42e None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:37.897669 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b2202-1f74-4786-9930-836e6c4cf42e None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:37.897669 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b2202-1f74-4786-9930-836e6c4cf42e None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:37.898935 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 641/1293] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:29:37.904028 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8603e-4823-459f-99e5-36e9ec9d1a67 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:37.904409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8603e-4823-459f-99e5-36e9ec9d1a67 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:37.904582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8603e-4823-459f-99e5-36e9ec9d1a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:37.921265 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-5de8603e-4823-459f-99e5-36e9ec9d1a67 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:29:37.922473 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-5de8603e-4823-459f-99e5-36e9ec9d1a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:29:37.951342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:37.990157 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 653/1294] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:29:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:29:38.013644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b2202-1f74-4786-9930-836e6c4cf42e None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:38.052662 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:38.131052 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b2202-1f74-4786-9930-836e6c4cf42e None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:38.164071 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd85c1-cda6-4f61-9b8a-e8a9b2404ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:38.164730 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd85c1-cda6-4f61-9b8a-e8a9b2404ae7 None admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:38.165015 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd85c1-cda6-4f61-9b8a-e8a9b2404ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:38.165319 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd85c1-cda6-4f61-9b8a-e8a9b2404ae7 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:38.166060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd85c1-cda6-4f61-9b8a-e8a9b2404ae7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:38.167431 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd85c1-cda6-4f61-9b8a-e8a9b2404ae7 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:38.180417 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 642/1295] 10.222.0.71 () {60 vars in 1141 bytes} [Thu Jul 21 20:29:38 2022] POST /identity/v3/services => generated 246 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:29:40.209951 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fb77a-e5d5-41b2-90cc-2bfa9d0d4e23 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:40.210507 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fb77a-e5d5-41b2-90cc-2bfa9d0d4e23 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:40.210507 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fb77a-e5d5-41b2-90cc-2bfa9d0d4e23 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:40.211850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 654/1296] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:29:40.217221 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25829d57-e1e8-4f79-a0f0-5c2d1a33e866 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:40.217620 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25829d57-e1e8-4f79-a0f0-5c2d1a33e866 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:40.217797 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25829d57-e1e8-4f79-a0f0-5c2d1a33e866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:40.234490 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-25829d57-e1e8-4f79-a0f0-5c2d1a33e866 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:29:40.235675 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-25829d57-e1e8-4f79-a0f0-5c2d1a33e866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:29:40.306824 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 643/1297] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:29:40 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:29:40.329815 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fb77a-e5d5-41b2-90cc-2bfa9d0d4e23 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:40.431366 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fb77a-e5d5-41b2-90cc-2bfa9d0d4e23 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:40.464593 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c29c1-a786-401d-bfc1-5c24f44d7ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:40.465348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c29c1-a786-401d-bfc1-5c24f44d7ca3 None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:40.465626 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c29c1-a786-401d-bfc1-5c24f44d7ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:40.465894 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c29c1-a786-401d-bfc1-5c24f44d7ca3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:40.466536 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c29c1-a786-401d-bfc1-5c24f44d7ca3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:40.467905 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c29c1-a786-401d-bfc1-5c24f44d7ca3 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:40.479660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-929c29c1-a786-401d-bfc1-5c24f44d7ca3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 20:29:40.479660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:40.479660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:29:40.479660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:29:40.479660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:29:40.479660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:29:40.479660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:29:40.479660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:29:40.479660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 20:29:40.479660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:40.479660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:29:40.479660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:40.479660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:40.479660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:40.479660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:40.479660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:40.479660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:40.480853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:40.480853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:40.480853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:40.480853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:40.480853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:40.480853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:40.480853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:29:40.480853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:29:40.480853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:29:40.480853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:29:40.480853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:40.480853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:40.480853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:40.480853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:40.480853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:40.480853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:40.482009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:40.482009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:40.482009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:40.482009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:40.482009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:40.482009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:40.482009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:40.482009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:40.482009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:40.482009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:40.482009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:29:40.482009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:29:40.482009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 20:29:40.482009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:40.482009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 655/1298] 10.222.0.71 () {56 vars in 1117 bytes} [Thu Jul 21 20:29:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:40.487039 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25829d57-e1e8-4f79-a0f0-5c2d1a33e866 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:40.507691 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:40.587064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25829d57-e1e8-4f79-a0f0-5c2d1a33e866 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:40.621179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49135c8-8049-41d7-8b41-5a1599d244c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:40.621829 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49135c8-8049-41d7-8b41-5a1599d244c1 None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:40.622108 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49135c8-8049-41d7-8b41-5a1599d244c1 None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:40.622379 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49135c8-8049-41d7-8b41-5a1599d244c1 None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:40.623081 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49135c8-8049-41d7-8b41-5a1599d244c1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:40.623387 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49135c8-8049-41d7-8b41-5a1599d244c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:40.624744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49135c8-8049-41d7-8b41-5a1599d244c1 None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:40.630325 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 644/1299] 10.222.0.71 () {56 vars in 1116 bytes} [Thu Jul 21 20:29:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:40.636516 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c29c1-a786-401d-bfc1-5c24f44d7ca3 None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:40.668000 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:40.736212 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c29c1-a786-401d-bfc1-5c24f44d7ca3 None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:40.769662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684e81d-765a-4a0f-8eba-0be835b0613c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:40.770274 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684e81d-765a-4a0f-8eba-0be835b0613c None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:40.770551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684e81d-765a-4a0f-8eba-0be835b0613c None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:40.770827 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684e81d-765a-4a0f-8eba-0be835b0613c None admin] PATH_INFO: `/v3/endpoints` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:40.771605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e684e81d-765a-4a0f-8eba-0be835b0613c None admin] RBAC: Adding query filter params (interface=public, service_id=1290adab7a85492999e37ee1999bfa10, region_id=RegionOne) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:40.771912 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e684e81d-765a-4a0f-8eba-0be835b0613c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:40.773327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e684e81d-765a-4a0f-8eba-0be835b0613c None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:40.778334 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 656/1300] 10.222.0.71 () {56 vars in 1251 bytes} [Thu Jul 21 20:29:40 2022] GET /identity/v3/endpoints?service_id=1290adab7a85492999e37ee1999bfa10&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:40.784099 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49135c8-8049-41d7-8b41-5a1599d244c1 None admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:40.865115 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:40.883681 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49135c8-8049-41d7-8b41-5a1599d244c1 None admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:40.917095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50d907-4b3a-40d4-bba9-41f8e311ba3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:40.917771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50d907-4b3a-40d4-bba9-41f8e311ba3f None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:40.918048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50d907-4b3a-40d4-bba9-41f8e311ba3f None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:40.918322 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50d907-4b3a-40d4-bba9-41f8e311ba3f None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:40.919008 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50d907-4b3a-40d4-bba9-41f8e311ba3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:40.920363 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50d907-4b3a-40d4-bba9-41f8e311ba3f None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:40.927689 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 645/1301] 10.222.0.71 () {56 vars in 1087 bytes} [Thu Jul 21 20:29:40 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:29:42.969260 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2479b0-46f3-4b0c-afd8-87e2d7a0e5d3 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:42.970090 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2479b0-46f3-4b0c-afd8-87e2d7a0e5d3 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:42.970090 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2479b0-46f3-4b0c-afd8-87e2d7a0e5d3 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:42.971449 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 657/1302] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:29:42.977190 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6415a49-8395-4303-bf09-3ead162d45d6 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:42.977601 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6415a49-8395-4303-bf09-3ead162d45d6 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:42.977743 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6415a49-8395-4303-bf09-3ead162d45d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:42.994060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-d6415a49-8395-4303-bf09-3ead162d45d6 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:29:42.995244 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d6415a49-8395-4303-bf09-3ead162d45d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:29:43.059709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 646/1303] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:29:42 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:29:43.083813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2479b0-46f3-4b0c-afd8-87e2d7a0e5d3 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:43.176782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:43.184320 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2479b0-46f3-4b0c-afd8-87e2d7a0e5d3 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:43.222144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087327ce-3341-428d-8b6f-1ca31412ee4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:43.223406 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087327ce-3341-428d-8b6f-1ca31412ee4d None admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:43.223916 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087327ce-3341-428d-8b6f-1ca31412ee4d None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:43.224418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087327ce-3341-428d-8b6f-1ca31412ee4d None admin] PATH_INFO: `/v3/services/placement` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:43.225153 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087327ce-3341-428d-8b6f-1ca31412ee4d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:43.227579 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087327ce-3341-428d-8b6f-1ca31412ee4d None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:43.240343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application [None req-087327ce-3341-428d-8b6f-1ca31412ee4d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 20:29:43.240343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:43.240343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:29:43.240343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:29:43.240343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:29:43.240343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:29:43.240343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:29:43.240343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:29:43.240343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 20:29:43.240343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:43.240343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:29:43.240343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:43.240343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:29:43.240343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:29:43.240343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:29:43.240343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:29:43.240343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:29:43.241601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:29:43.241601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:29:43.241601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:29:43.241601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:29:43.241601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:29:43.241601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:29:43.241601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:29:43.241601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:29:43.241601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:29:43.241601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:29:43.241601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:29:43.241601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:29:43.241601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:29:43.241601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:29:43.241601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:29:43.241601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:29:43.242686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:29:43.242686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application with Lock( Jul 21 20:29:43.242686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:29:43.242686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:29:43.242686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:29:43.242686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:29:43.242686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:29:43.242686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:29:43.242686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:29:43.242686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:29:43.242686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:29:43.242686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:29:43.242686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 20:29:43.242686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: ERROR keystone.server.flask.application Jul 21 20:29:43.242686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 658/1304] 10.222.0.71 () {56 vars in 1117 bytes} [Thu Jul 21 20:29:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:29:43.247717 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6415a49-8395-4303-bf09-3ead162d45d6 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:43.342164 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:43.365754 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6415a49-8395-4303-bf09-3ead162d45d6 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:43.400550 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e81ee6-1e58-43eb-8e40-e151d7734deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:43.401311 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e81ee6-1e58-43eb-8e40-e151d7734deb None admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:43.401595 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e81ee6-1e58-43eb-8e40-e151d7734deb None admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:43.401868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e81ee6-1e58-43eb-8e40-e151d7734deb None admin] PATH_INFO: `/v3/services` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:43.402576 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e81ee6-1e58-43eb-8e40-e151d7734deb None admin] RBAC: Adding query filter params (name=placement) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:29:43.402883 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e81ee6-1e58-43eb-8e40-e151d7734deb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:43.404240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e81ee6-1e58-43eb-8e40-e151d7734deb None admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:43.409297 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 647/1305] 10.222.0.71 () {56 vars in 1116 bytes} [Thu Jul 21 20:29:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:29:43.415237 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087327ce-3341-428d-8b6f-1ca31412ee4d None admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:29:43.517877 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087327ce-3341-428d-8b6f-1ca31412ee4d None admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:29:43.552156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01a09e-c9dc-4526-b90e-1ae57318db96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:29:43.552810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01a09e-c9dc-4526-b90e-1ae57318db96 None admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:29:43.553117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01a09e-c9dc-4526-b90e-1ae57318db96 None admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:29:43.553399 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01a09e-c9dc-4526-b90e-1ae57318db96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:29:43.554146 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01a09e-c9dc-4526-b90e-1ae57318db96 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:29:43.555522 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01a09e-c9dc-4526-b90e-1ae57318db96 None admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:29:43.574453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 659/1306] 10.222.0.71 () {60 vars in 1144 bytes} [Thu Jul 21 20:29:43 2022] POST /identity/v3/endpoints => generated 331 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:29:53.406310 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=30 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:29:53.569963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:30:48.048110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec08c7e-6c99-499d-ac93-8fd4e05b91ef None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:30:48.048972 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec08c7e-6c99-499d-ac93-8fd4e05b91ef None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:30:48.048972 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec08c7e-6c99-499d-ac93-8fd4e05b91ef None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:30:48.054542 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 648/1307] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:30:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:30:48.057271 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149977c-a812-47d5-812f-526398c1c3cc None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:30:48.058708 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149977c-a812-47d5-812f-526398c1c3cc None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:30:48.059453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149977c-a812-47d5-812f-526398c1c3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:30:48.356764 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-6149977c-a812-47d5-812f-526398c1c3cc None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:30:48.359395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-6149977c-a812-47d5-812f-526398c1c3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:30:48.558662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 660/1308] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 503 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:30:48.613354 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290545a1-5df6-49c3-a34c-fb9f957e71ab None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:30:48.613885 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290545a1-5df6-49c3-a34c-fb9f957e71ab None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:30:48.614313 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290545a1-5df6-49c3-a34c-fb9f957e71ab None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:30:48.616123 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 649/1309] 10.222.0.71 () {54 vars in 965 bytes} [Thu Jul 21 20:30:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:30:48.622719 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebf9a6-206e-44f2-bc43-f9e1770d1e26 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:30:48.623156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebf9a6-206e-44f2-bc43-f9e1770d1e26 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:30:48.623542 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebf9a6-206e-44f2-bc43-f9e1770d1e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:30:48.720367 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-e6ebf9a6-206e-44f2-bc43-f9e1770d1e26 None None] MFA Rules not processed for user `03eee8ef92314296b7d5c14c52f0d080`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:30:48.722123 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-e6ebf9a6-206e-44f2-bc43-f9e1770d1e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:30:48.745103 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:30:48.817260 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290545a1-5df6-49c3-a34c-fb9f957e71ab None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:30:48.819258 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-290545a1-5df6-49c3-a34c-fb9f957e71ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:30:48.822986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 661/1310] 10.222.0.71 () {58 vars in 1062 bytes} [Thu Jul 21 20:30:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:30:49.018280 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290545a1-5df6-49c3-a34c-fb9f957e71ab None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:30:49.118175 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a72286-3994-47ef-b8e9-7a25d8055bb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:30:49.119000 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a72286-3994-47ef-b8e9-7a25d8055bb5 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:30:49.119405 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a72286-3994-47ef-b8e9-7a25d8055bb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:30:49.119789 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a72286-3994-47ef-b8e9-7a25d8055bb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:30:49.121817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-87a72286-3994-47ef-b8e9-7a25d8055bb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:30:49.208078 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:30:49.278013 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a72286-3994-47ef-b8e9-7a25d8055bb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:30:49.279480 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a72286-3994-47ef-b8e9-7a25d8055bb5 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:30:49.303244 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 650/1311] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:30:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:30:51.778892 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e623f-3ad3-4528-9a51-0caf81285937 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:30:51.779496 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e623f-3ad3-4528-9a51-0caf81285937 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:30:51.779496 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e623f-3ad3-4528-9a51-0caf81285937 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:30:51.780730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 662/1312] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:30:51.786400 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9503599-bb87-446d-a597-aebdea26ad52 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:30:51.786814 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9503599-bb87-446d-a597-aebdea26ad52 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:30:51.786957 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9503599-bb87-446d-a597-aebdea26ad52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:30:51.877637 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-b9503599-bb87-446d-a597-aebdea26ad52 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:30:51.878868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-b9503599-bb87-446d-a597-aebdea26ad52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:30:51.988253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 651/1313] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:30:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:30:52.043678 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa5299-09cf-4195-bc90-2a88c2920107 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:30:52.043895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa5299-09cf-4195-bc90-2a88c2920107 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:30:52.044137 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa5299-09cf-4195-bc90-2a88c2920107 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:30:52.045603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 663/1314] 10.222.0.71 () {54 vars in 965 bytes} [Thu Jul 21 20:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:30:52.052494 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d7597-e6de-4a7b-9511-57e869550c3e None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:30:52.052787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d7597-e6de-4a7b-9511-57e869550c3e None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:30:52.053117 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d7597-e6de-4a7b-9511-57e869550c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:30:52.064095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:30:52.136624 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-6a9d7597-e6de-4a7b-9511-57e869550c3e None None] MFA Rules not processed for user `03eee8ef92314296b7d5c14c52f0d080`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:30:52.137928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-6a9d7597-e6de-4a7b-9511-57e869550c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:30:52.218474 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 652/1315] 10.222.0.71 () {58 vars in 1062 bytes} [Thu Jul 21 20:30:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:30:52.228013 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa5299-09cf-4195-bc90-2a88c2920107 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:30:52.229628 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-76fa5299-09cf-4195-bc90-2a88c2920107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:30:52.342690 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa5299-09cf-4195-bc90-2a88c2920107 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:30:52.418100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:30:52.421031 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839a0c1-2e7b-48fa-9a31-637f2b7a0790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:30:52.421872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839a0c1-2e7b-48fa-9a31-637f2b7a0790 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:30:52.422219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839a0c1-2e7b-48fa-9a31-637f2b7a0790 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:30:52.422642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839a0c1-2e7b-48fa-9a31-637f2b7a0790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:30:52.424595 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-1839a0c1-2e7b-48fa-9a31-637f2b7a0790 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:30:52.536126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839a0c1-2e7b-48fa-9a31-637f2b7a0790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:30:52.537609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839a0c1-2e7b-48fa-9a31-637f2b7a0790 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:30:52.556642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 664/1316] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:30:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:30:55.298292 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445b07a-512c-40ff-b68d-2c17e934ad0e None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:30:55.298891 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445b07a-512c-40ff-b68d-2c17e934ad0e None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:30:55.298891 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445b07a-512c-40ff-b68d-2c17e934ad0e None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:30:55.300272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 653/1317] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:30:55.306145 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d34e62-4ae3-4695-bcc3-2f5486d62a53 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:30:55.306717 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d34e62-4ae3-4695-bcc3-2f5486d62a53 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:30:55.306717 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d34e62-4ae3-4695-bcc3-2f5486d62a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:30:55.383676 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-f8d34e62-4ae3-4695-bcc3-2f5486d62a53 None None] MFA Rules not processed for user `89583f5f42ab49058e780d3e5c44662c`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:30:55.384931 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f8d34e62-4ae3-4695-bcc3-2f5486d62a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:30:55.437385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:30:55.487547 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 665/1318] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:30:55 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:30:55.528750 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445b07a-512c-40ff-b68d-2c17e934ad0e None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:30:55.530365 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-1445b07a-512c-40ff-b68d-2c17e934ad0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:30:55.644166 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445b07a-512c-40ff-b68d-2c17e934ad0e None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:30:55.647048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:30:55.720231 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f3488-c070-4ad6-bddb-707ee9b34bc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:30:55.721117 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f3488-c070-4ad6-bddb-707ee9b34bc3 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:30:55.721413 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f3488-c070-4ad6-bddb-707ee9b34bc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:30:55.721696 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f3488-c070-4ad6-bddb-707ee9b34bc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:30:55.723500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-0a5f3488-c070-4ad6-bddb-707ee9b34bc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:30:55.838679 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f3488-c070-4ad6-bddb-707ee9b34bc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:30:55.840129 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f3488-c070-4ad6-bddb-707ee9b34bc3 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:30:55.859276 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 654/1319] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:30:55 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:30:58.558462 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255915c-fb77-48dd-bee4-de5fbb5b82e7 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:30:58.559061 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255915c-fb77-48dd-bee4-de5fbb5b82e7 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:30:58.559193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255915c-fb77-48dd-bee4-de5fbb5b82e7 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:30:58.560861 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 666/1320] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:30:58.569872 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1798a-9879-4ae8-aa84-fe4a9706f556 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:30:58.570422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1798a-9879-4ae8-aa84-fe4a9706f556 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:30:58.570885 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1798a-9879-4ae8-aa84-fe4a9706f556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:30:58.635900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:30:58.665469 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-d7e1798a-9879-4ae8-aa84-fe4a9706f556 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:30:58.666723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d7e1798a-9879-4ae8-aa84-fe4a9706f556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:30:58.778677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 655/1321] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:30:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:30:58.820240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255915c-fb77-48dd-bee4-de5fbb5b82e7 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:30:58.821877 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-7255915c-fb77-48dd-bee4-de5fbb5b82e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:30:58.944588 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255915c-fb77-48dd-bee4-de5fbb5b82e7 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:30:59.036591 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847adcc3-6a2a-49f0-a3a7-45d31f1b7f6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:30:59.037278 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847adcc3-6a2a-49f0-a3a7-45d31f1b7f6d service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:30:59.037559 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847adcc3-6a2a-49f0-a3a7-45d31f1b7f6d service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:30:59.037833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847adcc3-6a2a-49f0-a3a7-45d31f1b7f6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:30:59.039710 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-847adcc3-6a2a-49f0-a3a7-45d31f1b7f6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:30:59.048604 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:30:59.187449 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847adcc3-6a2a-49f0-a3a7-45d31f1b7f6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:30:59.188945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847adcc3-6a2a-49f0-a3a7-45d31f1b7f6d service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:30:59.211042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 667/1322] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:30:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:31:02.325193 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2964fa3-1ea4-4d68-9eef-73cc34c1a795 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:02.325753 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2964fa3-1ea4-4d68-9eef-73cc34c1a795 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:02.326347 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2964fa3-1ea4-4d68-9eef-73cc34c1a795 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:02.329108 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 656/1323] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:31:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:02.335566 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe40764-0f63-4b2a-900c-0453c323c603 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:02.336333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe40764-0f63-4b2a-900c-0453c323c603 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:02.336333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe40764-0f63-4b2a-900c-0453c323c603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:02.419236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-cbe40764-0f63-4b2a-900c-0453c323c603 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:02.420512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-cbe40764-0f63-4b2a-900c-0453c323c603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:02.512001 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:02.560852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 668/1324] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:31:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:31:02.600637 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2964fa3-1ea4-4d68-9eef-73cc34c1a795 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:31:02.602745 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d2964fa3-1ea4-4d68-9eef-73cc34c1a795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:02.748818 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2964fa3-1ea4-4d68-9eef-73cc34c1a795 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:31:02.763750 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:02.848625 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310da76-dff7-4a7e-ab77-a1d418b142b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:31:02.849257 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310da76-dff7-4a7e-ab77-a1d418b142b5 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:02.849545 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310da76-dff7-4a7e-ab77-a1d418b142b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:02.849823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310da76-dff7-4a7e-ab77-a1d418b142b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:02.851567 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-b310da76-dff7-4a7e-ab77-a1d418b142b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:02.982641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310da76-dff7-4a7e-ab77-a1d418b142b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:31:02.984128 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310da76-dff7-4a7e-ab77-a1d418b142b5 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:31:03.008636 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 657/1325] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:31:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:31:05.201487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785f400-4882-4691-8fa2-6c85f5009ed2 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:05.202203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785f400-4882-4691-8fa2-6c85f5009ed2 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:05.202203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785f400-4882-4691-8fa2-6c85f5009ed2 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:05.203318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 669/1326] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:31:05.210416 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ee0bb-1049-4934-ad80-147c2ac61fd1 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:05.210921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ee0bb-1049-4934-ad80-147c2ac61fd1 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:05.210921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ee0bb-1049-4934-ad80-147c2ac61fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:05.264553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:05.284206 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-a46ee0bb-1049-4934-ad80-147c2ac61fd1 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:05.285438 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-a46ee0bb-1049-4934-ad80-147c2ac61fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:05.385696 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 658/1327] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:31:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:31:05.421614 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785f400-4882-4691-8fa2-6c85f5009ed2 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:31:05.423212 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-5785f400-4882-4691-8fa2-6c85f5009ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:05.538939 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785f400-4882-4691-8fa2-6c85f5009ed2 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:31:05.625294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab943c31-711f-45c3-8e4f-a118bec95d3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:31:05.625951 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab943c31-711f-45c3-8e4f-a118bec95d3b service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:05.626233 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab943c31-711f-45c3-8e4f-a118bec95d3b service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:05.626504 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab943c31-711f-45c3-8e4f-a118bec95d3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:05.628832 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-ab943c31-711f-45c3-8e4f-a118bec95d3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:05.648241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:05.743602 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab943c31-711f-45c3-8e4f-a118bec95d3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:31:05.745057 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab943c31-711f-45c3-8e4f-a118bec95d3b service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:31:05.763036 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 670/1328] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:31:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:31:07.891566 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21cdbb-84d8-4940-91ed-a9c2524938e9 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:07.892131 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21cdbb-84d8-4940-91ed-a9c2524938e9 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:07.892279 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21cdbb-84d8-4940-91ed-a9c2524938e9 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:07.893804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 659/1329] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:31:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:07.899868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec487a0-3ebe-437d-ae2c-5501f5b88c65 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:07.900378 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec487a0-3ebe-437d-ae2c-5501f5b88c65 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:07.900566 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec487a0-3ebe-437d-ae2c-5501f5b88c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:07.975873 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-aec487a0-3ebe-437d-ae2c-5501f5b88c65 None None] MFA Rules not processed for user `89583f5f42ab49058e780d3e5c44662c`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:07.977017 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-aec487a0-3ebe-437d-ae2c-5501f5b88c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:08.045188 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:08.083003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 671/1330] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:31:07 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:31:08.120726 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21cdbb-84d8-4940-91ed-a9c2524938e9 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:31:08.122299 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-ba21cdbb-84d8-4940-91ed-a9c2524938e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:08.229142 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21cdbb-84d8-4940-91ed-a9c2524938e9 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:31:08.250923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:08.308620 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797e2cb-5e56-4615-840d-1436c266484f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:31:08.309308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797e2cb-5e56-4615-840d-1436c266484f service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:08.309592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797e2cb-5e56-4615-840d-1436c266484f service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:08.309867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797e2cb-5e56-4615-840d-1436c266484f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:08.311554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c797e2cb-5e56-4615-840d-1436c266484f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:08.439865 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797e2cb-5e56-4615-840d-1436c266484f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:31:08.441391 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797e2cb-5e56-4615-840d-1436c266484f service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:31:08.464942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 660/1331] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:31:08 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:31:10.982520 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fca0a-db25-4ed2-8b95-00ce6cb7eefe None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:10.983102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fca0a-db25-4ed2-8b95-00ce6cb7eefe None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:10.983102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fca0a-db25-4ed2-8b95-00ce6cb7eefe None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:10.984479 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 672/1332] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:10.989724 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1075ce5-7e0c-47c4-9b77-d3ebe065534c None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:10.990116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1075ce5-7e0c-47c4-9b77-d3ebe065534c None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:10.990293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1075ce5-7e0c-47c4-9b77-d3ebe065534c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:11.052364 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:11.055579 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-d1075ce5-7e0c-47c4-9b77-d3ebe065534c None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:11.057161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d1075ce5-7e0c-47c4-9b77-d3ebe065534c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:11.156164 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 661/1333] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:31:11.193959 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fca0a-db25-4ed2-8b95-00ce6cb7eefe None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:31:11.195678 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-fa8fca0a-db25-4ed2-8b95-00ce6cb7eefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:11.300140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fca0a-db25-4ed2-8b95-00ce6cb7eefe None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:31:11.374534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae11bf2-58b3-4d60-955f-d909fda45f2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:31:11.375182 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae11bf2-58b3-4d60-955f-d909fda45f2f service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:11.375463 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae11bf2-58b3-4d60-955f-d909fda45f2f service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:11.375737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae11bf2-58b3-4d60-955f-d909fda45f2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:11.378037 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-8ae11bf2-58b3-4d60-955f-d909fda45f2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:11.404982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:11.495134 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae11bf2-58b3-4d60-955f-d909fda45f2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:31:11.496594 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae11bf2-58b3-4d60-955f-d909fda45f2f service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:31:11.516854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 673/1334] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:31:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:31:13.659841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21acb2-3fe5-4852-9020-5001f59ed067 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:13.660434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21acb2-3fe5-4852-9020-5001f59ed067 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:13.660434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21acb2-3fe5-4852-9020-5001f59ed067 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:13.661813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 662/1335] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:13.667320 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fc768-0f65-493d-b374-822d03f6def3 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:13.667775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fc768-0f65-493d-b374-822d03f6def3 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:13.667869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fc768-0f65-493d-b374-822d03f6def3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:13.735079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-798fc768-0f65-493d-b374-822d03f6def3 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:13.736194 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-798fc768-0f65-493d-b374-822d03f6def3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:13.812041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:13.836718 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 674/1336] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:31:13.878362 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21acb2-3fe5-4852-9020-5001f59ed067 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:31:13.879940 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-be21acb2-3fe5-4852-9020-5001f59ed067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:13.990686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21acb2-3fe5-4852-9020-5001f59ed067 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:31:14.020796 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:14.069761 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5a9f6-88c3-4bf3-b52d-16ce06f70635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:31:14.070428 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5a9f6-88c3-4bf3-b52d-16ce06f70635 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:14.070715 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5a9f6-88c3-4bf3-b52d-16ce06f70635 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:14.070994 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5a9f6-88c3-4bf3-b52d-16ce06f70635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:14.072709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-88a5a9f6-88c3-4bf3-b52d-16ce06f70635 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:14.195582 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5a9f6-88c3-4bf3-b52d-16ce06f70635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:31:14.197045 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5a9f6-88c3-4bf3-b52d-16ce06f70635 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:31:14.218075 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 663/1337] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:31:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:31:16.780943 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3c280-8003-4f3c-9fdd-e77f308a3dd9 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:16.781519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3c280-8003-4f3c-9fdd-e77f308a3dd9 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:16.781519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3c280-8003-4f3c-9fdd-e77f308a3dd9 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:16.782907 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 675/1338] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:16.788646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986fab6-3f46-4b4b-a303-a88bbc058759 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:16.789087 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986fab6-3f46-4b4b-a303-a88bbc058759 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:16.789220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986fab6-3f46-4b4b-a303-a88bbc058759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:16.852217 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-c986fab6-3f46-4b4b-a303-a88bbc058759 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:16.853472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c986fab6-3f46-4b4b-a303-a88bbc058759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:16.861538 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:16.947908 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 664/1339] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:31:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:31:16.984732 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3c280-8003-4f3c-9fdd-e77f308a3dd9 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:31:16.986304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-d1d3c280-8003-4f3c-9fdd-e77f308a3dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:17.094313 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3c280-8003-4f3c-9fdd-e77f308a3dd9 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:31:17.172129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e232ce-a1ae-4aa7-96aa-784ab5548394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:31:17.172788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e232ce-a1ae-4aa7-96aa-784ab5548394 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:17.173091 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e232ce-a1ae-4aa7-96aa-784ab5548394 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:17.173387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e232ce-a1ae-4aa7-96aa-784ab5548394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:17.175768 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-21e232ce-a1ae-4aa7-96aa-784ab5548394 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:17.214761 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:17.290949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e232ce-a1ae-4aa7-96aa-784ab5548394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:31:17.292398 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e232ce-a1ae-4aa7-96aa-784ab5548394 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:31:17.310986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 676/1340] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:31:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:31:21.471109 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e9b3b-3630-49f2-b304-0a4d86f46681 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:21.471576 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e9b3b-3630-49f2-b304-0a4d86f46681 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:21.471576 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e9b3b-3630-49f2-b304-0a4d86f46681 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:21.473280 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 665/1341] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:31:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:21.478703 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cf6d8-e47a-4ece-a1a8-6bb8a3606a54 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:21.479175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cf6d8-e47a-4ece-a1a8-6bb8a3606a54 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:21.479269 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cf6d8-e47a-4ece-a1a8-6bb8a3606a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:21.547156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-b43cf6d8-e47a-4ece-a1a8-6bb8a3606a54 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:21.548364 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-b43cf6d8-e47a-4ece-a1a8-6bb8a3606a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:21.636441 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:21.648557 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 677/1342] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:31:21.689486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e9b3b-3630-49f2-b304-0a4d86f46681 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:31:21.691108 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-6b0e9b3b-3630-49f2-b304-0a4d86f46681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:21.795028 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e9b3b-3630-49f2-b304-0a4d86f46681 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:31:21.835259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:21.873300 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed5af1-32d8-4b4f-aae6-a3184bd029d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:31:21.873962 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed5af1-32d8-4b4f-aae6-a3184bd029d1 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:21.874248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed5af1-32d8-4b4f-aae6-a3184bd029d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:21.874547 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed5af1-32d8-4b4f-aae6-a3184bd029d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:21.876208 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-92ed5af1-32d8-4b4f-aae6-a3184bd029d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:21.986765 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed5af1-32d8-4b4f-aae6-a3184bd029d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:31:21.988211 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed5af1-32d8-4b4f-aae6-a3184bd029d1 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:31:22.006503 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 666/1343] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:31:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:31:24.708789 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87a64b-2ab5-4020-b053-8837c77b1298 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:24.709483 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87a64b-2ab5-4020-b053-8837c77b1298 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:24.709483 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87a64b-2ab5-4020-b053-8837c77b1298 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:24.710990 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 678/1344] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:24.716736 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6a9a0-a716-4c4e-aa8a-ed2152006d6b None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:24.717348 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6a9a0-a716-4c4e-aa8a-ed2152006d6b None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:24.717348 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6a9a0-a716-4c4e-aa8a-ed2152006d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:24.791063 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-e4d6a9a0-a716-4c4e-aa8a-ed2152006d6b None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:24.792303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-e4d6a9a0-a716-4c4e-aa8a-ed2152006d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:24.813583 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:24.899998 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 667/1345] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:31:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:31:24.937753 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87a64b-2ab5-4020-b053-8837c77b1298 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:31:24.939936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-cd87a64b-2ab5-4020-b053-8837c77b1298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:25.061673 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87a64b-2ab5-4020-b053-8837c77b1298 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:31:25.141470 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec92f9-5f7e-4e37-9114-57c1d57e30d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:31:25.142034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec92f9-5f7e-4e37-9114-57c1d57e30d6 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:25.142868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec92f9-5f7e-4e37-9114-57c1d57e30d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:25.143146 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec92f9-5f7e-4e37-9114-57c1d57e30d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:25.144824 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-fcec92f9-5f7e-4e37-9114-57c1d57e30d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:25.190828 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:25.259647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec92f9-5f7e-4e37-9114-57c1d57e30d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:31:25.261123 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec92f9-5f7e-4e37-9114-57c1d57e30d6 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:31:25.281336 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 679/1346] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:31:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:31:29.606749 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00ebd4-01a1-48ae-be64-5f1aaed46b01 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:29.607299 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00ebd4-01a1-48ae-be64-5f1aaed46b01 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:29.607299 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00ebd4-01a1-48ae-be64-5f1aaed46b01 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:29.608834 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 668/1347] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:29.614406 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88ea91-3b22-4b37-9315-0190b2b83893 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:29.614881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88ea91-3b22-4b37-9315-0190b2b83893 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:29.614987 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88ea91-3b22-4b37-9315-0190b2b83893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:29.685406 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-9a88ea91-3b22-4b37-9315-0190b2b83893 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:29.686628 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-9a88ea91-3b22-4b37-9315-0190b2b83893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:29.788109 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:29.791547 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 680/1348] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:31:29.830676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00ebd4-01a1-48ae-be64-5f1aaed46b01 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:31:29.831960 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-7e00ebd4-01a1-48ae-be64-5f1aaed46b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:29.940133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00ebd4-01a1-48ae-be64-5f1aaed46b01 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:31:30.000765 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:30.029439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c5c44-ea2a-428e-8f93-1a3619f1aed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:31:30.030093 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c5c44-ea2a-428e-8f93-1a3619f1aed2 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:30.030377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c5c44-ea2a-428e-8f93-1a3619f1aed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:30.030686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c5c44-ea2a-428e-8f93-1a3619f1aed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:30.032413 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-be0c5c44-ea2a-428e-8f93-1a3619f1aed2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:30.149904 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c5c44-ea2a-428e-8f93-1a3619f1aed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:31:30.151431 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c5c44-ea2a-428e-8f93-1a3619f1aed2 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:31:30.170968 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 669/1349] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:31:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:31:35.144850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a6c8a-394c-4080-b67d-097c7154922c None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:35.145487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a6c8a-394c-4080-b67d-097c7154922c None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:35.145487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a6c8a-394c-4080-b67d-097c7154922c None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:35.147178 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 681/1350] 10.222.0.71 () {54 vars in 921 bytes} [Thu Jul 21 20:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:35.152060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b93311-3b72-452e-8034-2c0f2b4d6dab None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:35.152473 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b93311-3b72-452e-8034-2c0f2b4d6dab None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:35.152664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b93311-3b72-452e-8034-2c0f2b4d6dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:35.251961 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-21b93311-3b72-452e-8034-2c0f2b4d6dab None None] MFA Rules not processed for user `364e80ec79b94812a1df76c62c2b13a0`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:35.253098 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-21b93311-3b72-452e-8034-2c0f2b4d6dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:35.276353 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:35.380679 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 670/1351] 10.222.0.71 () {58 vars in 1018 bytes} [Thu Jul 21 20:31:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:31:39.460289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20327c1c-9874-42c6-86e2-bf835f624b3b None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:39.460289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20327c1c-9874-42c6-86e2-bf835f624b3b None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:39.460857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20327c1c-9874-42c6-86e2-bf835f624b3b None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:39.461846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 682/1352] 10.222.0.71 () {54 vars in 921 bytes} [Thu Jul 21 20:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:39.467436 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac72c4-a76e-46e5-838d-29420c0526bf None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:39.467853 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac72c4-a76e-46e5-838d-29420c0526bf None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:39.467996 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac72c4-a76e-46e5-838d-29420c0526bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:39.555260 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-1fac72c4-a76e-46e5-838d-29420c0526bf None None] MFA Rules not processed for user `364e80ec79b94812a1df76c62c2b13a0`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:39.556527 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-1fac72c4-a76e-46e5-838d-29420c0526bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:39.635054 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:39.674196 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 671/1353] 10.222.0.71 () {58 vars in 1018 bytes} [Thu Jul 21 20:31:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:31:41.156940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df539260-073c-4e6a-9cea-c6940a105126 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:41.157737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df539260-073c-4e6a-9cea-c6940a105126 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:41.157737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df539260-073c-4e6a-9cea-c6940a105126 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:41.159503 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 683/1354] 10.222.0.71 () {54 vars in 921 bytes} [Thu Jul 21 20:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:41.164313 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417e726-3815-4887-ab32-4fac19b876db None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:41.164745 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417e726-3815-4887-ab32-4fac19b876db None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:41.165001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417e726-3815-4887-ab32-4fac19b876db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:41.241890 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-c417e726-3815-4887-ab32-4fac19b876db None None] MFA Rules not processed for user `364e80ec79b94812a1df76c62c2b13a0`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:41.243107 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c417e726-3815-4887-ab32-4fac19b876db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:41.353793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 672/1355] 10.222.0.71 () {58 vars in 1018 bytes} [Thu Jul 21 20:31:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:31:44.787666 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d71dc9-4240-4282-96be-8911210bc9ef None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:44.788169 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d71dc9-4240-4282-96be-8911210bc9ef None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:44.788169 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d71dc9-4240-4282-96be-8911210bc9ef None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:44.790767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 684/1356] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:31:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:44.796006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b55de-9020-41fd-95a2-b1e35f5d5246 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:44.796471 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b55de-9020-41fd-95a2-b1e35f5d5246 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:44.796575 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b55de-9020-41fd-95a2-b1e35f5d5246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:44.849672 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:44.886509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-085b55de-9020-41fd-95a2-b1e35f5d5246 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:44.888115 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-085b55de-9020-41fd-95a2-b1e35f5d5246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:45.025691 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 673/1357] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:31:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:31:45.078431 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1fc47-95e3-4199-8473-99bd06053d59 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:45.079036 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1fc47-95e3-4199-8473-99bd06053d59 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:45.079564 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1fc47-95e3-4199-8473-99bd06053d59 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:45.081809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 685/1358] 10.222.0.71 () {54 vars in 962 bytes} [Thu Jul 21 20:31:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:45.091623 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f75b86-06af-49c7-985c-29d5741589a3 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:45.091952 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f75b86-06af-49c7-985c-29d5741589a3 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:45.092183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f75b86-06af-49c7-985c-29d5741589a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:45.181294 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-a8f75b86-06af-49c7-985c-29d5741589a3 None None] MFA Rules not processed for user `6fbc5240f5214aaea4418052b94739e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:45.182916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-a8f75b86-06af-49c7-985c-29d5741589a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:45.240237 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:45.317760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 674/1359] 10.222.0.71 () {58 vars in 1059 bytes} [Thu Jul 21 20:31:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:31:45.326152 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1fc47-95e3-4199-8473-99bd06053d59 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:31:45.327808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-eec1fc47-95e3-4199-8473-99bd06053d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:45.485198 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1fc47-95e3-4199-8473-99bd06053d59 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:31:45.559784 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:45.604277 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7b263-ab1a-4fb9-83bd-8a164a7ae115 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:31:45.604928 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7b263-ab1a-4fb9-83bd-8a164a7ae115 service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:45.605232 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7b263-ab1a-4fb9-83bd-8a164a7ae115 service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:45.605505 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7b263-ab1a-4fb9-83bd-8a164a7ae115 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:45.607137 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f4b7b263-ab1a-4fb9-83bd-8a164a7ae115 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:45.701411 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7b263-ab1a-4fb9-83bd-8a164a7ae115 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:31:45.703014 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7b263-ab1a-4fb9-83bd-8a164a7ae115 service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:31:45.722528 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 686/1360] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:31:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:31:48.015262 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdd9b9-8b5a-4855-bd0c-a9969fb40a45 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:48.015786 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdd9b9-8b5a-4855-bd0c-a9969fb40a45 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:48.015786 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdd9b9-8b5a-4855-bd0c-a9969fb40a45 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:48.017433 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 675/1361] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:48.023063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf348b-e66e-4c5e-b0a4-ec441d1c1690 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:48.023784 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf348b-e66e-4c5e-b0a4-ec441d1c1690 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:48.023784 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf348b-e66e-4c5e-b0a4-ec441d1c1690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:48.094482 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-1ddf348b-e66e-4c5e-b0a4-ec441d1c1690 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:48.096202 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-1ddf348b-e66e-4c5e-b0a4-ec441d1c1690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:48.134807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:48.202002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 687/1362] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:31:48.247759 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdd9b9-8b5a-4855-bd0c-a9969fb40a45 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:31:48.249470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-25cdd9b9-8b5a-4855-bd0c-a9969fb40a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:48.426087 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdd9b9-8b5a-4855-bd0c-a9969fb40a45 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:31:48.436671 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:48.594874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be933687-30c0-4c08-adaa-1520530d01b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:31:48.595562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be933687-30c0-4c08-adaa-1520530d01b8 service nova] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:48.595839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be933687-30c0-4c08-adaa-1520530d01b8 service nova] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:48.596111 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be933687-30c0-4c08-adaa-1520530d01b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:48.597866 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-be933687-30c0-4c08-adaa-1520530d01b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:48.694735 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be933687-30c0-4c08-adaa-1520530d01b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:31:48.696354 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be933687-30c0-4c08-adaa-1520530d01b8 service nova] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:31:48.715900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 676/1363] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:31:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:31:51.118832 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc365a-09a1-4ad6-ac65-62d270a552c7 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:51.119359 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc365a-09a1-4ad6-ac65-62d270a552c7 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:51.119457 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc365a-09a1-4ad6-ac65-62d270a552c7 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:51.120770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 688/1364] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:51.126588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062d304-e324-4cf8-95e1-d9ae1ca2332d None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:51.127058 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062d304-e324-4cf8-95e1-d9ae1ca2332d None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:51.127157 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062d304-e324-4cf8-95e1-d9ae1ca2332d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:51.142817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:51.204929 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-4062d304-e324-4cf8-95e1-d9ae1ca2332d None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:51.206335 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-4062d304-e324-4cf8-95e1-d9ae1ca2332d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:51.318025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 677/1365] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:31:51.365504 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35cd28-0268-4346-b7e8-c6441f0f9df1 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:51.366040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35cd28-0268-4346-b7e8-c6441f0f9df1 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:51.366500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35cd28-0268-4346-b7e8-c6441f0f9df1 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:51.368196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 689/1366] 10.222.0.71 () {54 vars in 962 bytes} [Thu Jul 21 20:31:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:51.373833 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874857b-7734-45c6-a769-d6958d1d62a1 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:51.374298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874857b-7734-45c6-a769-d6958d1d62a1 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:51.374748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874857b-7734-45c6-a769-d6958d1d62a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:51.460695 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-5874857b-7734-45c6-a769-d6958d1d62a1 None None] MFA Rules not processed for user `6fbc5240f5214aaea4418052b94739e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:51.461953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-5874857b-7734-45c6-a769-d6958d1d62a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:51.504548 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:51.601242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 678/1367] 10.222.0.71 () {58 vars in 1059 bytes} [Thu Jul 21 20:31:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:31:51.609104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35cd28-0268-4346-b7e8-c6441f0f9df1 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:31:51.610677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-8c35cd28-0268-4346-b7e8-c6441f0f9df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:51.766924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35cd28-0268-4346-b7e8-c6441f0f9df1 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:31:51.789135 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:51.890491 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62e5b7-fada-4b44-a968-1f1b71774b72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:31:51.891135 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62e5b7-fada-4b44-a968-1f1b71774b72 service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:51.891420 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62e5b7-fada-4b44-a968-1f1b71774b72 service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:51.892335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62e5b7-fada-4b44-a968-1f1b71774b72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:51.894161 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3e62e5b7-fada-4b44-a968-1f1b71774b72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:51.990875 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62e5b7-fada-4b44-a968-1f1b71774b72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:31:51.992517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62e5b7-fada-4b44-a968-1f1b71774b72 service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:31:52.011400 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 690/1368] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:31:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:31:53.050918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be175f-6651-4a06-a358-cc97db7feefc None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:53.051583 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be175f-6651-4a06-a358-cc97db7feefc None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:53.051583 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be175f-6651-4a06-a358-cc97db7feefc None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:53.053560 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 679/1369] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:31:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:53.059539 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5a90e-f9d3-48e4-8950-65fd813ce7d6 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:53.060005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5a90e-f9d3-48e4-8950-65fd813ce7d6 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:53.060195 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5a90e-f9d3-48e4-8950-65fd813ce7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:53.099886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:53.142713 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-7cc5a90e-f9d3-48e4-8950-65fd813ce7d6 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:53.144127 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-7cc5a90e-f9d3-48e4-8950-65fd813ce7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:53.277189 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 691/1370] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:31:53.630667 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40072e-3b98-42c0-b6fe-9f17ce890ebe None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:53.630954 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40072e-3b98-42c0-b6fe-9f17ce890ebe None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:53.631235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40072e-3b98-42c0-b6fe-9f17ce890ebe None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:53.633215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 680/1371] 10.222.0.71 () {54 vars in 964 bytes} [Thu Jul 21 20:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:53.639065 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325778c-5c42-49e2-b612-6d78007cdc4d None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:53.639456 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325778c-5c42-49e2-b612-6d78007cdc4d None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:53.639628 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325778c-5c42-49e2-b612-6d78007cdc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:53.730338 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-9325778c-5c42-49e2-b612-6d78007cdc4d None None] MFA Rules not processed for user `34db429cd31549f78daf5b1cbf6f0bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:53.731783 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-9325778c-5c42-49e2-b612-6d78007cdc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:53.784405 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:53.817628 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 692/1372] 10.222.0.71 () {58 vars in 1061 bytes} [Thu Jul 21 20:31:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:31:53.826211 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40072e-3b98-42c0-b6fe-9f17ce890ebe None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:31:53.827826 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-5d40072e-3b98-42c0-b6fe-9f17ce890ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:53.956169 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40072e-3b98-42c0-b6fe-9f17ce890ebe None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:31:54.003857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:54.046544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589b234-b2f2-4955-a7bc-3b5777929110 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:31:54.047525 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589b234-b2f2-4955-a7bc-3b5777929110 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:54.047658 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589b234-b2f2-4955-a7bc-3b5777929110 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:54.048593 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589b234-b2f2-4955-a7bc-3b5777929110 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:54.050572 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c589b234-b2f2-4955-a7bc-3b5777929110 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:54.156313 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85947dae-ec50-48c3-a555-980645f13636 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:54.156923 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85947dae-ec50-48c3-a555-980645f13636 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:54.157010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85947dae-ec50-48c3-a555-980645f13636 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:54.160262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 693/1373] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:31:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:31:54.164874 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972d773-3bee-486a-96a5-4412b65ede24 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:54.165630 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972d773-3bee-486a-96a5-4412b65ede24 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:54.165793 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972d773-3bee-486a-96a5-4412b65ede24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:54.188570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589b234-b2f2-4955-a7bc-3b5777929110 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:31:54.190489 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589b234-b2f2-4955-a7bc-3b5777929110 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:31:54.212278 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 681/1374] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:31:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:31:54.243300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-3972d773-3bee-486a-96a5-4412b65ede24 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:54.244549 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3972d773-3bee-486a-96a5-4412b65ede24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:54.353540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 694/1375] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:31:54.399802 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589b234-b2f2-4955-a7bc-3b5777929110 service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:31:54.401384 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c589b234-b2f2-4955-a7bc-3b5777929110 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:54.494394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:54.534595 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589b234-b2f2-4955-a7bc-3b5777929110 service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:31:54.648262 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cc064-3873-4857-a1f8-e9f000e6d397 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:31:54.648912 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cc064-3873-4857-a1f8-e9f000e6d397 service nova] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:54.649222 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cc064-3873-4857-a1f8-e9f000e6d397 service nova] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:54.649492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cc064-3873-4857-a1f8-e9f000e6d397 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:54.651124 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-f07cc064-3873-4857-a1f8-e9f000e6d397 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:54.733263 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:54.746907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07cc064-3873-4857-a1f8-e9f000e6d397 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:31:54.748528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07cc064-3873-4857-a1f8-e9f000e6d397 service nova] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:31:54.767284 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 682/1376] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:31:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:31:56.878185 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e63129-df50-49e2-b4f0-7facec4b5bd8 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:56.878730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e63129-df50-49e2-b4f0-7facec4b5bd8 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:56.878730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e63129-df50-49e2-b4f0-7facec4b5bd8 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:56.880129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 695/1377] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:56.885582 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c97a8f-ac3c-4184-a438-8b408fb6f2e6 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:56.886004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c97a8f-ac3c-4184-a438-8b408fb6f2e6 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:56.886182 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c97a8f-ac3c-4184-a438-8b408fb6f2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:56.966715 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-a7c97a8f-ac3c-4184-a438-8b408fb6f2e6 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:56.967870 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-a7c97a8f-ac3c-4184-a438-8b408fb6f2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:57.066093 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 683/1378] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:31:57.113273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e63129-df50-49e2-b4f0-7facec4b5bd8 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:31:57.114807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-b4e63129-df50-49e2-b4f0-7facec4b5bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:57.173375 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:57.289869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e63129-df50-49e2-b4f0-7facec4b5bd8 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:31:57.433418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5227df0-741c-46d0-b921-ea3dddfaca13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:31:57.433989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5227df0-741c-46d0-b921-ea3dddfaca13 service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:57.434269 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5227df0-741c-46d0-b921-ea3dddfaca13 service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:57.434540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5227df0-741c-46d0-b921-ea3dddfaca13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:57.436230 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-e5227df0-741c-46d0-b921-ea3dddfaca13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:57.448439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:57.537974 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5227df0-741c-46d0-b921-ea3dddfaca13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:31:57.539842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5227df0-741c-46d0-b921-ea3dddfaca13 service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:31:57.560699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 696/1379] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:31:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:31:59.698932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6cc4e-e515-4398-9d11-b72359ddc6c3 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:59.699577 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6cc4e-e515-4398-9d11-b72359ddc6c3 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:59.699577 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6cc4e-e515-4398-9d11-b72359ddc6c3 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:59.701108 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 684/1380] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:31:59.707028 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4932b1-fdf8-404c-8af3-284c03e99832 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:31:59.707451 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4932b1-fdf8-404c-8af3-284c03e99832 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:31:59.707535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4932b1-fdf8-404c-8af3-284c03e99832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:31:59.777490 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-de4932b1-fdf8-404c-8af3-284c03e99832 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:31:59.778703 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-de4932b1-fdf8-404c-8af3-284c03e99832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:31:59.869915 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:31:59.884276 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 697/1381] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:31:59.930460 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6cc4e-e515-4398-9d11-b72359ddc6c3 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:31:59.932388 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-e8b6cc4e-e515-4398-9d11-b72359ddc6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:00.010917 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:00.105442 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6cc4e-e515-4398-9d11-b72359ddc6c3 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:00.228153 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981bd83-16de-48f3-bb8d-27d107c22ebd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:00.229313 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981bd83-16de-48f3-bb8d-27d107c22ebd service nova] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:00.229708 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981bd83-16de-48f3-bb8d-27d107c22ebd service nova] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:00.230041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981bd83-16de-48f3-bb8d-27d107c22ebd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:00.232259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c981bd83-16de-48f3-bb8d-27d107c22ebd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:00.257168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:00.344569 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981bd83-16de-48f3-bb8d-27d107c22ebd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:00.346358 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981bd83-16de-48f3-bb8d-27d107c22ebd service nova] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:00.369602 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 685/1382] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:31:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:32:02.481990 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00bc93-bdad-46f8-be90-232d87a82d0d None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:02.482559 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00bc93-bdad-46f8-be90-232d87a82d0d None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:02.482685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00bc93-bdad-46f8-be90-232d87a82d0d None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:02.484469 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 698/1383] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:32:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:02.491334 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b5249-c2ce-4519-a973-59dd49669f12 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:02.491818 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b5249-c2ce-4519-a973-59dd49669f12 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:02.491913 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b5249-c2ce-4519-a973-59dd49669f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:02.558323 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-811b5249-c2ce-4519-a973-59dd49669f12 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:02.559557 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-811b5249-c2ce-4519-a973-59dd49669f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:02.659332 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:02.663113 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 686/1384] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:02.708029 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00bc93-bdad-46f8-be90-232d87a82d0d None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:02.709614 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-ab00bc93-bdad-46f8-be90-232d87a82d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:02.863905 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00bc93-bdad-46f8-be90-232d87a82d0d None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:02.932030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:02.994524 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd37d0-d737-4b40-98ff-a29debe910a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:02.995211 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd37d0-d737-4b40-98ff-a29debe910a4 service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:02.995506 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd37d0-d737-4b40-98ff-a29debe910a4 service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:02.995799 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd37d0-d737-4b40-98ff-a29debe910a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:02.997559 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-49fd37d0-d737-4b40-98ff-a29debe910a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:03.106753 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd37d0-d737-4b40-98ff-a29debe910a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:03.108458 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd37d0-d737-4b40-98ff-a29debe910a4 service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:03.130018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 699/1385] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:32:05.198669 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae673e65-8e60-4acc-8436-7cddcfeb7a5b None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:05.199642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae673e65-8e60-4acc-8436-7cddcfeb7a5b None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:05.199642 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae673e65-8e60-4acc-8436-7cddcfeb7a5b None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:05.200609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 687/1386] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:05.206498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acc665-fda8-4ba4-a1fe-9636108cd26c None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:05.207305 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acc665-fda8-4ba4-a1fe-9636108cd26c None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:05.207305 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acc665-fda8-4ba4-a1fe-9636108cd26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:05.289197 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-f4acc665-fda8-4ba4-a1fe-9636108cd26c None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:05.290299 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f4acc665-fda8-4ba4-a1fe-9636108cd26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:05.321836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:05.406476 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 700/1387] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:32:05.453859 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae673e65-8e60-4acc-8436-7cddcfeb7a5b None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:05.455650 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-ae673e65-8e60-4acc-8436-7cddcfeb7a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:05.619261 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae673e65-8e60-4acc-8436-7cddcfeb7a5b None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:05.696734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:05.752142 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0243aa1-7284-46b8-8a97-df2e4827baa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:05.752837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0243aa1-7284-46b8-8a97-df2e4827baa7 service nova] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:05.753249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0243aa1-7284-46b8-8a97-df2e4827baa7 service nova] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:05.753451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0243aa1-7284-46b8-8a97-df2e4827baa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:05.756477 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-e0243aa1-7284-46b8-8a97-df2e4827baa7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:05.867954 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0243aa1-7284-46b8-8a97-df2e4827baa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:05.869740 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0243aa1-7284-46b8-8a97-df2e4827baa7 service nova] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:05.888939 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 688/1388] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:32:08.249906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07438e-01a5-4779-9fd4-a23ded115c4d None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:08.250441 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07438e-01a5-4779-9fd4-a23ded115c4d None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:08.250441 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07438e-01a5-4779-9fd4-a23ded115c4d None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:08.252387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 701/1389] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:32:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:08.257707 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90897764-3d4d-41bc-af24-3699871b2772 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:08.258126 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90897764-3d4d-41bc-af24-3699871b2772 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:08.258268 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90897764-3d4d-41bc-af24-3699871b2772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:08.326071 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-90897764-3d4d-41bc-af24-3699871b2772 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:08.327305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-90897764-3d4d-41bc-af24-3699871b2772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:08.384974 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:08.450280 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 689/1390] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:08.495142 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07438e-01a5-4779-9fd4-a23ded115c4d None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:08.496839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-bd07438e-01a5-4779-9fd4-a23ded115c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:08.651238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07438e-01a5-4779-9fd4-a23ded115c4d None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:08.660923 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:08.765788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1cfba-db49-401f-830c-f0fdd52e3456 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:08.766383 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1cfba-db49-401f-830c-f0fdd52e3456 service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:08.767191 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1cfba-db49-401f-830c-f0fdd52e3456 service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:08.767468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1cfba-db49-401f-830c-f0fdd52e3456 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:08.769156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-99e1cfba-db49-401f-830c-f0fdd52e3456 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:08.860404 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e1cfba-db49-401f-830c-f0fdd52e3456 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:08.862055 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e1cfba-db49-401f-830c-f0fdd52e3456 service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:08.880409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 702/1391] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:32:09.491111 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cd34a-5380-46fe-b63f-557ac0e370fa None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:09.491656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cd34a-5380-46fe-b63f-557ac0e370fa None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:09.491656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cd34a-5380-46fe-b63f-557ac0e370fa None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:09.493102 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 690/1392] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:09.498393 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cdeeb-447d-461f-9c9f-ec5c33e4d080 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:09.498870 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cdeeb-447d-461f-9c9f-ec5c33e4d080 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:09.498983 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cdeeb-447d-461f-9c9f-ec5c33e4d080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:09.512892 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:09.565850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-9b5cdeeb-447d-461f-9c9f-ec5c33e4d080 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:09.567047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-9b5cdeeb-447d-461f-9c9f-ec5c33e4d080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:09.676985 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 703/1393] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:32:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:09.732632 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23980bb-fc18-4af5-919f-590b7819cc41 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:09.732920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23980bb-fc18-4af5-919f-590b7819cc41 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:09.733235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23980bb-fc18-4af5-919f-590b7819cc41 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:09.734609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 691/1394] 10.222.0.71 () {54 vars in 966 bytes} [Thu Jul 21 20:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:09.740468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccffe9-ad2f-45dd-8b2d-0fe22b7397cc None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:09.740828 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccffe9-ad2f-45dd-8b2d-0fe22b7397cc None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:09.741234 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccffe9-ad2f-45dd-8b2d-0fe22b7397cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:09.825860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-64ccffe9-ad2f-45dd-8b2d-0fe22b7397cc None None] MFA Rules not processed for user `ee2460e6a7e34a6881984e4f7ed294c1`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:09.827072 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-64ccffe9-ad2f-45dd-8b2d-0fe22b7397cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:09.863613 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:09.916510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 704/1395] 10.222.0.71 () {58 vars in 1063 bytes} [Thu Jul 21 20:32:09 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:09.924045 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23980bb-fc18-4af5-919f-590b7819cc41 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:09.925621 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-b23980bb-fc18-4af5-919f-590b7819cc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:10.033513 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23980bb-fc18-4af5-919f-590b7819cc41 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:10.097195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:10.109944 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2906cb0-1361-48a3-adb6-69919258ffd8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2460e6a7e34a6881984e4f7ed294c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:10.110629 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2906cb0-1361-48a3-adb6-69919258ffd8 service glance] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:10.110907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2906cb0-1361-48a3-adb6-69919258ffd8 service glance] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:10.111182 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2906cb0-1361-48a3-adb6-69919258ffd8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:10.112886 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d2906cb0-1361-48a3-adb6-69919258ffd8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:10.224166 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2906cb0-1361-48a3-adb6-69919258ffd8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:10.225651 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2906cb0-1361-48a3-adb6-69919258ffd8 service glance] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:10.243559 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 692/1396] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:32:10.970236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f993c-2011-47a2-b181-80b2cb3d2659 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:10.970706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f993c-2011-47a2-b181-80b2cb3d2659 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:10.970706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f993c-2011-47a2-b181-80b2cb3d2659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:10.974009 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf690dc-a5d9-4973-bad3-a291848b49c2 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:10.974467 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf690dc-a5d9-4973-bad3-a291848b49c2 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:10.974566 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf690dc-a5d9-4973-bad3-a291848b49c2 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:10.975979 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 693/1397] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:32:10.980978 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad3970-8baa-4313-9885-c5a9c857e887 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:10.981302 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad3970-8baa-4313-9885-c5a9c857e887 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:10.981611 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad3970-8baa-4313-9885-c5a9c857e887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:11.040086 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-fd5f993c-2011-47a2-b181-80b2cb3d2659 None None] MFA Rules not processed for user `b94e634e8855486a9ce14d3de9d986fb`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:11.041412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-fd5f993c-2011-47a2-b181-80b2cb3d2659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:11.050511 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-09ad3970-8baa-4313-9885-c5a9c857e887 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:11.051740 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-09ad3970-8baa-4313-9885-c5a9c857e887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:11.093943 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:11.130349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 705/1398] 10.222.0.71 () {58 vars in 1009 bytes} [Thu Jul 21 20:32:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:11.155835 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 694/1399] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:32:11.178295 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ce810-e5c5-4425-aa02-fb64c76a019a None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:11.178608 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ce810-e5c5-4425-aa02-fb64c76a019a None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:11.178887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ce810-e5c5-4425-aa02-fb64c76a019a None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:11.180267 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 706/1400] 10.222.0.71 () {54 vars in 943 bytes} [Thu Jul 21 20:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:32:11.186704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f4830-5494-42fc-b98d-7ace4867e6bb None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:11.187056 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f4830-5494-42fc-b98d-7ace4867e6bb None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:11.187376 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f4830-5494-42fc-b98d-7ace4867e6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:11.201715 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ce810-e5c5-4425-aa02-fb64c76a019a None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:11.203226 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-d73ce810-e5c5-4425-aa02-fb64c76a019a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:11.257413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:11.263746 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-dc9f4830-5494-42fc-b98d-7ace4867e6bb None None] MFA Rules not processed for user `3babb078203647e2b3ef3b2787b84e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:11.264981 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-dc9f4830-5494-42fc-b98d-7ace4867e6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:11.355373 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ce810-e5c5-4425-aa02-fb64c76a019a None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:11.373695 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 695/1401] 10.222.0.71 () {58 vars in 1040 bytes} [Thu Jul 21 20:32:11 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:11.383454 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f4830-5494-42fc-b98d-7ace4867e6bb None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:11.389690 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:11.394198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f4830-5494-42fc-b98d-7ace4867e6bb None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:11.479449 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665e9b8-438d-4e4a-8997-bbb453727332 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:11.480096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665e9b8-438d-4e4a-8997-bbb453727332 service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:11.480376 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665e9b8-438d-4e4a-8997-bbb453727332 service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:11.480651 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665e9b8-438d-4e4a-8997-bbb453727332 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:11.482396 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-e665e9b8-438d-4e4a-8997-bbb453727332 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:11.495587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:11.506580 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884eafc-de50-440a-84d4-3a0bc93f862f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3babb078203647e2b3ef3b2787b84e3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:11.507238 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884eafc-de50-440a-84d4-3a0bc93f862f service swift] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:11.507572 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884eafc-de50-440a-84d4-3a0bc93f862f service swift] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:11.507797 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884eafc-de50-440a-84d4-3a0bc93f862f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:11.509592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-8884eafc-de50-440a-84d4-3a0bc93f862f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:11.581675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e665e9b8-438d-4e4a-8997-bbb453727332 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:11.583299 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e665e9b8-438d-4e4a-8997-bbb453727332 service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:11.599639 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884eafc-de50-440a-84d4-3a0bc93f862f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:11.601322 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884eafc-de50-440a-84d4-3a0bc93f862f service swift] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:11.608539 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 707/1402] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:32:11.610033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 696/1403] 10.222.0.71 () {60 vars in 1384 bytes} [Thu Jul 21 20:32:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 20:32:13.739163 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad2b03-c046-45e3-9498-7d278237f698 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:13.739665 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad2b03-c046-45e3-9498-7d278237f698 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:13.739665 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad2b03-c046-45e3-9498-7d278237f698 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:13.741320 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 708/1404] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:13.746793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92953516-7b29-41a8-9f03-9d39a6181186 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:13.747305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92953516-7b29-41a8-9f03-9d39a6181186 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:13.747417 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92953516-7b29-41a8-9f03-9d39a6181186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:13.790769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:13.821996 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-92953516-7b29-41a8-9f03-9d39a6181186 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:13.823317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-92953516-7b29-41a8-9f03-9d39a6181186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:13.959383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 697/1405] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:14.006757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad2b03-c046-45e3-9498-7d278237f698 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:14.008538 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-68ad2b03-c046-45e3-9498-7d278237f698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:14.217462 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:14.232614 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad2b03-c046-45e3-9498-7d278237f698 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:14.449054 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4562b-b238-4820-8acf-e30978118395 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:14.450240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4562b-b238-4820-8acf-e30978118395 service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:14.450674 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4562b-b238-4820-8acf-e30978118395 service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:14.451105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4562b-b238-4820-8acf-e30978118395 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:14.452991 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f6f4562b-b238-4820-8acf-e30978118395 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:14.639317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4562b-b238-4820-8acf-e30978118395 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:14.641042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4562b-b238-4820-8acf-e30978118395 service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:14.654616 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:14.667429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 709/1406] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 661 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:32:16.851719 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843eb2c-0406-469a-84ed-487b093ce092 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:16.852271 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843eb2c-0406-469a-84ed-487b093ce092 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:16.852271 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843eb2c-0406-469a-84ed-487b093ce092 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:16.854246 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 698/1407] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:32:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:16.859653 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5ddf7-2d2f-484b-94c4-9e723f61d7f2 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:16.860684 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5ddf7-2d2f-484b-94c4-9e723f61d7f2 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:16.860790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5ddf7-2d2f-484b-94c4-9e723f61d7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:16.937526 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-92d5ddf7-2d2f-484b-94c4-9e723f61d7f2 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:16.938990 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-92d5ddf7-2d2f-484b-94c4-9e723f61d7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:17.060897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 710/1408] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:32:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:32:17.106739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843eb2c-0406-469a-84ed-487b093ce092 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:17.108273 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-8843eb2c-0406-469a-84ed-487b093ce092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:17.270665 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:17.323689 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843eb2c-0406-469a-84ed-487b093ce092 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:17.483323 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ddd64-76b8-4318-b2e4-d29c529b21fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:17.483981 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ddd64-76b8-4318-b2e4-d29c529b21fb service nova] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:17.484321 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ddd64-76b8-4318-b2e4-d29c529b21fb service nova] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:17.484610 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ddd64-76b8-4318-b2e4-d29c529b21fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:17.486848 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-f15ddd64-76b8-4318-b2e4-d29c529b21fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:17.584727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:17.610323 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ddd64-76b8-4318-b2e4-d29c529b21fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:17.611997 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ddd64-76b8-4318-b2e4-d29c529b21fb service nova] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:17.637480 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 699/1409] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:32:27.061165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=67 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:27.632835 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=13 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:34.899486 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bd7ae-344a-4a60-baae-92792f2cb5ac None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:34.900024 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bd7ae-344a-4a60-baae-92792f2cb5ac None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:34.900024 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bd7ae-344a-4a60-baae-92792f2cb5ac None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:34.901415 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 711/1410] 10.222.0.71 () {54 vars in 918 bytes} [Thu Jul 21 20:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:34.906698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c51c1a-b8b5-4c06-a924-1479026446df None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:34.907130 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c51c1a-b8b5-4c06-a924-1479026446df None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:34.907255 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c51c1a-b8b5-4c06-a924-1479026446df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:34.982390 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-82c51c1a-b8b5-4c06-a924-1479026446df None None] MFA Rules not processed for user `364e80ec79b94812a1df76c62c2b13a0`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:34.983610 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-82c51c1a-b8b5-4c06-a924-1479026446df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:35.090940 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 700/1411] 10.222.0.71 () {58 vars in 1015 bytes} [Thu Jul 21 20:32:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:35.106581 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279bd7c-e9cd-40ca-98be-3e3d7ee9f906 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:35.106872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279bd7c-e9cd-40ca-98be-3e3d7ee9f906 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:35.107153 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279bd7c-e9cd-40ca-98be-3e3d7ee9f906 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:35.108520 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 712/1412] 10.222.0.71 () {54 vars in 918 bytes} [Thu Jul 21 20:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:35.113724 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59fd2a-ad53-42bc-acbb-f05c5e58bacb None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:35.114018 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59fd2a-ad53-42bc-acbb-f05c5e58bacb None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:35.114294 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59fd2a-ad53-42bc-acbb-f05c5e58bacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:35.187577 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:35.190682 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-ca59fd2a-ad53-42bc-acbb-f05c5e58bacb None None] MFA Rules not processed for user `6fbc5240f5214aaea4418052b94739e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:35.192451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-ca59fd2a-ad53-42bc-acbb-f05c5e58bacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:35.309787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 701/1413] 10.222.0.71 () {58 vars in 1015 bytes} [Thu Jul 21 20:32:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:37.826703 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741ce0e-8f65-49d0-968b-4eea54b79746 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:37.827272 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741ce0e-8f65-49d0-968b-4eea54b79746 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:37.827272 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741ce0e-8f65-49d0-968b-4eea54b79746 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:37.828600 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:37.834860 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c09017-52f5-41a5-87f0-d91c0364c7e8 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:37.835348 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c09017-52f5-41a5-87f0-d91c0364c7e8 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:37.835453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c09017-52f5-41a5-87f0-d91c0364c7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:37.901790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-d2c09017-52f5-41a5-87f0-d91c0364c7e8 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:37.903006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d2c09017-52f5-41a5-87f0-d91c0364c7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:37.960921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:38.006077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:38.048980 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741ce0e-8f65-49d0-968b-4eea54b79746 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:38.050789 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-5741ce0e-8f65-49d0-968b-4eea54b79746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:38.216264 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741ce0e-8f65-49d0-968b-4eea54b79746 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:38.295965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:38.337047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f989c1b-ccc7-40a1-ba14-049876df3ab6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:38.337716 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f989c1b-ccc7-40a1-ba14-049876df3ab6 service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:38.337998 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f989c1b-ccc7-40a1-ba14-049876df3ab6 service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:38.338275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f989c1b-ccc7-40a1-ba14-049876df3ab6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:38.339930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-2f989c1b-ccc7-40a1-ba14-049876df3ab6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:38.429888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f989c1b-ccc7-40a1-ba14-049876df3ab6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:38.431499 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f989c1b-ccc7-40a1-ba14-049876df3ab6 service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:38.450093 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 714/1416] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:32:39.367129 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a188ba-1e42-4744-a4ff-c71ce25c053d None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:39.367792 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a188ba-1e42-4744-a4ff-c71ce25c053d None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:39.367792 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a188ba-1e42-4744-a4ff-c71ce25c053d None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:39.369212 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:32:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:39.374730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948119eb-b014-4d26-8966-dfaac4da69bf None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:39.375214 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948119eb-b014-4d26-8966-dfaac4da69bf None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:39.375318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948119eb-b014-4d26-8966-dfaac4da69bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:39.458030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-948119eb-b014-4d26-8966-dfaac4da69bf None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:39.459247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-948119eb-b014-4d26-8966-dfaac4da69bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:39.489949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:39.564733 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:32:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:39.721280 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a188ba-1e42-4744-a4ff-c71ce25c053d None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:39.722878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-f6a188ba-1e42-4744-a4ff-c71ce25c053d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:39.876984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a188ba-1e42-4744-a4ff-c71ce25c053d None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:39.920724 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:40.006671 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63c2d9-a537-413f-ae2a-c688c5e1b52f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:40.007260 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63c2d9-a537-413f-ae2a-c688c5e1b52f service nova] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:40.007545 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63c2d9-a537-413f-ae2a-c688c5e1b52f service nova] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:40.007823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63c2d9-a537-413f-ae2a-c688c5e1b52f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:40.009617 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-8d63c2d9-a537-413f-ae2a-c688c5e1b52f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:40.105362 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63c2d9-a537-413f-ae2a-c688c5e1b52f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:40.106996 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63c2d9-a537-413f-ae2a-c688c5e1b52f service nova] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:40.126157 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 704/1419] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:32:41.014179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbc0a5-4786-414f-bbf6-58651906844d None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:41.014757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbc0a5-4786-414f-bbf6-58651906844d None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:41.014757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbc0a5-4786-414f-bbf6-58651906844d None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:41.016117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:32:41.021542 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba7017-a949-4d5e-8441-65817827698f None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:41.022054 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba7017-a949-4d5e-8441-65817827698f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:41.022165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba7017-a949-4d5e-8441-65817827698f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:41.076912 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:41.099774 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-1dba7017-a949-4d5e-8441-65817827698f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:41.101004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-1dba7017-a949-4d5e-8441-65817827698f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:41.202880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:41.355487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbc0a5-4786-414f-bbf6-58651906844d None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:41.357028 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-17cbc0a5-4786-414f-bbf6-58651906844d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:41.512712 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbc0a5-4786-414f-bbf6-58651906844d None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:41.526096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:41.632227 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903b432-bc1b-4a26-9d84-8eeb4f453023 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:41.632878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903b432-bc1b-4a26-9d84-8eeb4f453023 service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:41.633184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903b432-bc1b-4a26-9d84-8eeb4f453023 service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:41.633458 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903b432-bc1b-4a26-9d84-8eeb4f453023 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:41.635137 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-1903b432-bc1b-4a26-9d84-8eeb4f453023 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:41.727115 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903b432-bc1b-4a26-9d84-8eeb4f453023 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:41.728658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903b432-bc1b-4a26-9d84-8eeb4f453023 service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:41.747759 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 717/1422] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:32:42.629564 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6079cd8-b705-49b1-a5f8-0a52c4f82f73 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:42.630155 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6079cd8-b705-49b1-a5f8-0a52c4f82f73 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:42.630155 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6079cd8-b705-49b1-a5f8-0a52c4f82f73 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:42.631491 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:42.637540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bd789-691e-4b99-b397-18039c7b4a50 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:42.638067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bd789-691e-4b99-b397-18039c7b4a50 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:42.638067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bd789-691e-4b99-b397-18039c7b4a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:42.657308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:42.718123 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-7a9bd789-691e-4b99-b397-18039c7b4a50 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:42.719496 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-7a9bd789-691e-4b99-b397-18039c7b4a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:42.821334 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:42.973554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6079cd8-b705-49b1-a5f8-0a52c4f82f73 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:42.975098 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-b6079cd8-b705-49b1-a5f8-0a52c4f82f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:43.097094 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:43.136354 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6079cd8-b705-49b1-a5f8-0a52c4f82f73 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:43.260684 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe01da-0032-47b4-808d-4804b70ec765 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:43.261274 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe01da-0032-47b4-808d-4804b70ec765 service nova] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:43.261562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe01da-0032-47b4-808d-4804b70ec765 service nova] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:43.261918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe01da-0032-47b4-808d-4804b70ec765 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:43.263791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-70fe01da-0032-47b4-808d-4804b70ec765 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:43.358952 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:43.375030 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe01da-0032-47b4-808d-4804b70ec765 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:43.377296 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe01da-0032-47b4-808d-4804b70ec765 service nova] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:43.397655 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 707/1425] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:32:44.302329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff951fdb-7394-4897-a002-5c02d3835666 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:44.303088 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff951fdb-7394-4897-a002-5c02d3835666 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:44.303088 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff951fdb-7394-4897-a002-5c02d3835666 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:44.304445 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:44.310035 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea8dc5-f52d-4090-adad-81d3ef594479 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:44.310592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea8dc5-f52d-4090-adad-81d3ef594479 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:44.310592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea8dc5-f52d-4090-adad-81d3ef594479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:44.392978 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-9fea8dc5-f52d-4090-adad-81d3ef594479 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:44.394230 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-9fea8dc5-f52d-4090-adad-81d3ef594479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:44.499900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:32:44.654145 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff951fdb-7394-4897-a002-5c02d3835666 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:44.655615 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-ff951fdb-7394-4897-a002-5c02d3835666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:44.748880 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:44.821030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff951fdb-7394-4897-a002-5c02d3835666 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:44.953735 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee486f74-f46c-40ba-bcd0-f454e2bcee40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:44.954547 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee486f74-f46c-40ba-bcd0-f454e2bcee40 service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:44.954826 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee486f74-f46c-40ba-bcd0-f454e2bcee40 service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:44.955104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee486f74-f46c-40ba-bcd0-f454e2bcee40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:44.956831 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-ee486f74-f46c-40ba-bcd0-f454e2bcee40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:45.010652 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:45.062034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee486f74-f46c-40ba-bcd0-f454e2bcee40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:45.063670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee486f74-f46c-40ba-bcd0-f454e2bcee40 service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:45.083597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 720/1428] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:32:45.908483 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de1c17-e47d-43c6-86ec-9785fed204b6 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:45.909137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de1c17-e47d-43c6-86ec-9785fed204b6 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:45.909137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de1c17-e47d-43c6-86ec-9785fed204b6 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:45.910486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:45.915769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb78511-8eff-49e5-becd-398325795356 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:45.916275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb78511-8eff-49e5-becd-398325795356 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:45.916377 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb78511-8eff-49e5-becd-398325795356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:45.997919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-8eb78511-8eff-49e5-becd-398325795356 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:45.999156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-8eb78511-8eff-49e5-becd-398325795356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:46.114418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:46.273601 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de1c17-e47d-43c6-86ec-9785fed204b6 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:46.275070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-74de1c17-e47d-43c6-86ec-9785fed204b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:46.330369 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:46.441206 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de1c17-e47d-43c6-86ec-9785fed204b6 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:46.558919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c90cc1-93d3-4a52-9b9d-b1dc1e07c4e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:46.559490 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c90cc1-93d3-4a52-9b9d-b1dc1e07c4e0 service nova] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:46.559775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c90cc1-93d3-4a52-9b9d-b1dc1e07c4e0 service nova] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:46.560038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c90cc1-93d3-4a52-9b9d-b1dc1e07c4e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:46.561892 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-44c90cc1-93d3-4a52-9b9d-b1dc1e07c4e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:46.569866 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:46.667698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c90cc1-93d3-4a52-9b9d-b1dc1e07c4e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:46.669366 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c90cc1-93d3-4a52-9b9d-b1dc1e07c4e0 service nova] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:46.689167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 710/1431] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:32:47.544448 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62957d-df30-4101-9f59-3d63893c4a79 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:47.545025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62957d-df30-4101-9f59-3d63893c4a79 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:47.545025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62957d-df30-4101-9f59-3d63893c4a79 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:47.546415 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:47.552500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b86c2-3e1b-4291-8260-8b9b51089e0f None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:47.553023 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b86c2-3e1b-4291-8260-8b9b51089e0f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:47.553175 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b86c2-3e1b-4291-8260-8b9b51089e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:47.637003 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-038b86c2-3e1b-4291-8260-8b9b51089e0f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:47.638286 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-038b86c2-3e1b-4291-8260-8b9b51089e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:47.715163 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:47.747568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:32:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:47.904712 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62957d-df30-4101-9f59-3d63893c4a79 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:47.906300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-0d62957d-df30-4101-9f59-3d63893c4a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:47.927104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:48.079555 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62957d-df30-4101-9f59-3d63893c4a79 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:48.176836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:48.206577 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93042a4-c55a-4cc6-9f8d-829682612c58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:48.207241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93042a4-c55a-4cc6-9f8d-829682612c58 service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:48.207526 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93042a4-c55a-4cc6-9f8d-829682612c58 service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:48.207806 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93042a4-c55a-4cc6-9f8d-829682612c58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:48.209740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-b93042a4-c55a-4cc6-9f8d-829682612c58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:48.313592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93042a4-c55a-4cc6-9f8d-829682612c58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:48.315856 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93042a4-c55a-4cc6-9f8d-829682612c58 service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:48.335595 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 723/1434] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:32:49.175945 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802be34-bd23-43f1-9895-3a312363595c None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:49.176524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802be34-bd23-43f1-9895-3a312363595c None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:49.176524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802be34-bd23-43f1-9895-3a312363595c None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:49.177942 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:49.183742 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51d9c9-02ba-4559-9810-4adbd5a4da61 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:49.184304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51d9c9-02ba-4559-9810-4adbd5a4da61 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:49.184304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51d9c9-02ba-4559-9810-4adbd5a4da61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:49.280904 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-3a51d9c9-02ba-4559-9810-4adbd5a4da61 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:49.282159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3a51d9c9-02ba-4559-9810-4adbd5a4da61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:49.330789 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:49.395858 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:32:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:32:49.552581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802be34-bd23-43f1-9895-3a312363595c None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:49.554281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-8802be34-bd23-43f1-9895-3a312363595c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:49.724826 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802be34-bd23-43f1-9895-3a312363595c None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:49.815435 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:49.881829 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66a811-5157-483c-a268-5af332a4fc42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:49.882506 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66a811-5157-483c-a268-5af332a4fc42 service nova] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:49.882786 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66a811-5157-483c-a268-5af332a4fc42 service nova] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:49.883067 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66a811-5157-483c-a268-5af332a4fc42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:49.884816 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-ce66a811-5157-483c-a268-5af332a4fc42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:49.989260 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66a811-5157-483c-a268-5af332a4fc42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:49.991000 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66a811-5157-483c-a268-5af332a4fc42 service nova] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:50.012836 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 713/1437] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:32:50.871892 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c795f1-1bd3-4e46-a25c-f8aa044d34ec None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:50.872623 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c795f1-1bd3-4e46-a25c-f8aa044d34ec None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:50.872623 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c795f1-1bd3-4e46-a25c-f8aa044d34ec None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:50.873915 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:50.879670 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79d221-cf9e-41ba-9c9f-84d1d2a51f70 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:50.880195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79d221-cf9e-41ba-9c9f-84d1d2a51f70 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:50.880316 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79d221-cf9e-41ba-9c9f-84d1d2a51f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:50.970899 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-fa79d221-cf9e-41ba-9c9f-84d1d2a51f70 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:50.972168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-fa79d221-cf9e-41ba-9c9f-84d1d2a51f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:50.974562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:51.082444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:32:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:51.236851 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c795f1-1bd3-4e46-a25c-f8aa044d34ec None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:51.238699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-59c795f1-1bd3-4e46-a25c-f8aa044d34ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:51.418300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c795f1-1bd3-4e46-a25c-f8aa044d34ec None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:51.450321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:51.555937 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e5450-b53d-42af-9178-f4a0ec483ae4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:51.556483 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e5450-b53d-42af-9178-f4a0ec483ae4 service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:51.556763 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e5450-b53d-42af-9178-f4a0ec483ae4 service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:51.557035 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e5450-b53d-42af-9178-f4a0ec483ae4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:51.558873 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-dc0e5450-b53d-42af-9178-f4a0ec483ae4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:51.663390 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e5450-b53d-42af-9178-f4a0ec483ae4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:51.665480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e5450-b53d-42af-9178-f4a0ec483ae4 service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:51.685244 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 726/1440] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:32:53.755192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8d12a-6079-4c2b-903b-954bddb33892 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:53.755737 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8d12a-6079-4c2b-903b-954bddb33892 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:53.755912 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8d12a-6079-4c2b-903b-954bddb33892 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:53.758223 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:32:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:53.764406 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8685d32-170a-4c4c-8650-946265ba197b None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:53.765014 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8685d32-170a-4c4c-8650-946265ba197b None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:53.765014 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8685d32-170a-4c4c-8650-946265ba197b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:53.808014 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:53.841938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-e8685d32-170a-4c4c-8650-946265ba197b None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:53.843171 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-e8685d32-170a-4c4c-8650-946265ba197b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:53.956285 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:32:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:53.998059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8d12a-6079-4c2b-903b-954bddb33892 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:53.999805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-23f8d12a-6079-4c2b-903b-954bddb33892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:54.119445 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8d12a-6079-4c2b-903b-954bddb33892 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:54.157478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:54.206451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a62f32-1008-4b09-a3a1-a91d80b3471e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:54.207119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a62f32-1008-4b09-a3a1-a91d80b3471e service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:54.207407 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a62f32-1008-4b09-a3a1-a91d80b3471e service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:54.207686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a62f32-1008-4b09-a3a1-a91d80b3471e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:54.209474 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-66a62f32-1008-4b09-a3a1-a91d80b3471e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:54.328440 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a62f32-1008-4b09-a3a1-a91d80b3471e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:54.330574 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a62f32-1008-4b09-a3a1-a91d80b3471e service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:54.350463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 716/1443] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:32:56.593646 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599ced2-df77-41ec-b785-fbc15eff2302 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:56.594184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599ced2-df77-41ec-b785-fbc15eff2302 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:56.594184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599ced2-df77-41ec-b785-fbc15eff2302 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:56.595593 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:56.600981 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a897e1d-21f5-4aa7-864f-4df9d394293a None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:56.601440 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a897e1d-21f5-4aa7-864f-4df9d394293a None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:56.601584 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a897e1d-21f5-4aa7-864f-4df9d394293a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:56.665372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-8a897e1d-21f5-4aa7-864f-4df9d394293a None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:56.666597 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-8a897e1d-21f5-4aa7-864f-4df9d394293a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:56.680101 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:56.767137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:56.805735 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599ced2-df77-41ec-b785-fbc15eff2302 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:56.807269 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a599ced2-df77-41ec-b785-fbc15eff2302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:56.910909 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599ced2-df77-41ec-b785-fbc15eff2302 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:56.922791 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:56.984288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d939c1-f9c3-40ae-842f-3fca62db96b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:56.984842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d939c1-f9c3-40ae-842f-3fca62db96b8 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:56.985145 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d939c1-f9c3-40ae-842f-3fca62db96b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:56.985418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d939c1-f9c3-40ae-842f-3fca62db96b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:56.988329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-06d939c1-f9c3-40ae-842f-3fca62db96b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:57.097846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d939c1-f9c3-40ae-842f-3fca62db96b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:32:57.099291 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d939c1-f9c3-40ae-842f-3fca62db96b8 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:32:57.117195 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 729/1446] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:32:59.574408 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382390ee-7191-405e-bbee-5f07a4f0d1c7 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:59.574941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382390ee-7191-405e-bbee-5f07a4f0d1c7 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:59.575067 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382390ee-7191-405e-bbee-5f07a4f0d1c7 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:59.576356 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:32:59.582631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9026b3a-5500-44cf-acfd-57364cf03a23 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:59.583082 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9026b3a-5500-44cf-acfd-57364cf03a23 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:59.583178 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9026b3a-5500-44cf-acfd-57364cf03a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:59.633947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:59.653100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-a9026b3a-5500-44cf-acfd-57364cf03a23 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:32:59.654337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a9026b3a-5500-44cf-acfd-57364cf03a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:59.752761 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:32:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:32:59.791306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382390ee-7191-405e-bbee-5f07a4f0d1c7 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:32:59.792837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-382390ee-7191-405e-bbee-5f07a4f0d1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:32:59.895874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382390ee-7191-405e-bbee-5f07a4f0d1c7 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:32:59.936199 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:32:59.971921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef13c63-d88c-405b-bd0a-50e4f6ef162f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:32:59.972573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef13c63-d88c-405b-bd0a-50e4f6ef162f service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:32:59.972878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef13c63-d88c-405b-bd0a-50e4f6ef162f service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:32:59.973182 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef13c63-d88c-405b-bd0a-50e4f6ef162f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:32:59.974852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-4ef13c63-d88c-405b-bd0a-50e4f6ef162f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:00.086189 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef13c63-d88c-405b-bd0a-50e4f6ef162f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:33:00.087535 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef13c63-d88c-405b-bd0a-50e4f6ef162f service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:33:00.105568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 719/1449] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:32:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:33:02.300049 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858440b-b59f-4612-9263-074978fb9952 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:02.300638 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858440b-b59f-4612-9263-074978fb9952 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:02.300638 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858440b-b59f-4612-9263-074978fb9952 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:02.302156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:33:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:33:02.308618 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af32ad-4372-4f55-990f-d0f29afeb2f4 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:02.309101 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af32ad-4372-4f55-990f-d0f29afeb2f4 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:02.309230 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af32ad-4372-4f55-990f-d0f29afeb2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:02.375768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-00af32ad-4372-4f55-990f-d0f29afeb2f4 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:33:02.377202 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-00af32ad-4372-4f55-990f-d0f29afeb2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:02.401144 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:02.487768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:33:02.528069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858440b-b59f-4612-9263-074978fb9952 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:33:02.529666 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-5858440b-b59f-4612-9263-074978fb9952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:02.648620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858440b-b59f-4612-9263-074978fb9952 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:33:02.670128 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:02.727185 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb8d55-b452-40c7-b93c-5bdc524e9b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:33:02.727841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb8d55-b452-40c7-b93c-5bdc524e9b17 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:02.728120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb8d55-b452-40c7-b93c-5bdc524e9b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:02.728397 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb8d55-b452-40c7-b93c-5bdc524e9b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:02.730134 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-e4cb8d55-b452-40c7-b93c-5bdc524e9b17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:02.843019 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb8d55-b452-40c7-b93c-5bdc524e9b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:33:02.844720 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb8d55-b452-40c7-b93c-5bdc524e9b17 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:33:02.862906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 732/1452] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:33:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:33:05.010375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09ac5a-0797-4751-875d-4d8da7520b22 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:05.010970 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09ac5a-0797-4751-875d-4d8da7520b22 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:05.010970 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09ac5a-0797-4751-875d-4d8da7520b22 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:05.012313 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:33:05.018888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff18d3-8815-46ac-a145-93da24690520 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:05.019471 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff18d3-8815-46ac-a145-93da24690520 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:05.019471 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff18d3-8815-46ac-a145-93da24690520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:05.092338 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:05.095968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-3eff18d3-8815-46ac-a145-93da24690520 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:33:05.097942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3eff18d3-8815-46ac-a145-93da24690520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:05.201063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:33:05.244804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09ac5a-0797-4751-875d-4d8da7520b22 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:33:05.247326 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-5e09ac5a-0797-4751-875d-4d8da7520b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:05.358707 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09ac5a-0797-4751-875d-4d8da7520b22 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:33:05.408169 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:05.433592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a305e-ce7c-4551-a398-9ef2384d4122 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:33:05.434249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a305e-ce7c-4551-a398-9ef2384d4122 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:05.434534 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a305e-ce7c-4551-a398-9ef2384d4122 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:05.434811 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a305e-ce7c-4551-a398-9ef2384d4122 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:05.436478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-970a305e-ce7c-4551-a398-9ef2384d4122 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:05.561526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a305e-ce7c-4551-a398-9ef2384d4122 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:33:05.563126 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a305e-ce7c-4551-a398-9ef2384d4122 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:33:05.583096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 722/1455] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:33:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:33:07.679704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f738c0f-a067-4bb6-9431-bda83db81c51 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:07.680296 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f738c0f-a067-4bb6-9431-bda83db81c51 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:07.680296 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f738c0f-a067-4bb6-9431-bda83db81c51 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:07.681662 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:33:07.686927 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce496b-20a6-45ac-b4f0-10eaa7f6b903 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:07.687356 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce496b-20a6-45ac-b4f0-10eaa7f6b903 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:07.687478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce496b-20a6-45ac-b4f0-10eaa7f6b903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:07.751945 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-a9ce496b-20a6-45ac-b4f0-10eaa7f6b903 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:33:07.753051 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-a9ce496b-20a6-45ac-b4f0-10eaa7f6b903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:07.781000 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:07.850083 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:33:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:33:07.888543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f738c0f-a067-4bb6-9431-bda83db81c51 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:33:07.890108 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-9f738c0f-a067-4bb6-9431-bda83db81c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:07.998435 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f738c0f-a067-4bb6-9431-bda83db81c51 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:33:08.027665 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:08.072946 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4e642-6936-4e7b-9abc-09fbb36b3920 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:33:08.073639 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4e642-6936-4e7b-9abc-09fbb36b3920 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:08.073920 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4e642-6936-4e7b-9abc-09fbb36b3920 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:08.074194 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4e642-6936-4e7b-9abc-09fbb36b3920 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:08.075846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-aba4e642-6936-4e7b-9abc-09fbb36b3920 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:08.185159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba4e642-6936-4e7b-9abc-09fbb36b3920 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:33:08.186644 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba4e642-6936-4e7b-9abc-09fbb36b3920 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:33:08.205535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 735/1458] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:33:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:33:10.306280 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09a447-22d1-42ec-9b6a-d4bc407e4b30 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:10.307031 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09a447-22d1-42ec-9b6a-d4bc407e4b30 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:10.307031 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09a447-22d1-42ec-9b6a-d4bc407e4b30 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:10.308232 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:33:10.314177 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0ae6a-68c0-4020-b093-e8b60ae0ea6b None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:10.314622 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0ae6a-68c0-4020-b093-e8b60ae0ea6b None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:10.314727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0ae6a-68c0-4020-b093-e8b60ae0ea6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:10.378828 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-25f0ae6a-68c0-4020-b093-e8b60ae0ea6b None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:33:10.380048 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-25f0ae6a-68c0-4020-b093-e8b60ae0ea6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:10.388610 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:10.481762 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:33:10.520240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09a447-22d1-42ec-9b6a-d4bc407e4b30 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:33:10.521851 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-be09a447-22d1-42ec-9b6a-d4bc407e4b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:10.625735 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09a447-22d1-42ec-9b6a-d4bc407e4b30 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:33:10.682020 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:10.701372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62590218-2565-459f-9724-6ddc59cf614b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:33:10.702023 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62590218-2565-459f-9724-6ddc59cf614b service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:10.702308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62590218-2565-459f-9724-6ddc59cf614b service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:10.702585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62590218-2565-459f-9724-6ddc59cf614b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:10.704532 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-62590218-2565-459f-9724-6ddc59cf614b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:10.812186 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62590218-2565-459f-9724-6ddc59cf614b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:33:10.814379 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62590218-2565-459f-9724-6ddc59cf614b service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:33:10.832385 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 725/1461] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:33:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:33:12.937845 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536ae24-de20-469d-bb36-e1c1bb68ed7f None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:12.938450 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536ae24-de20-469d-bb36-e1c1bb68ed7f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:12.938450 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536ae24-de20-469d-bb36-e1c1bb68ed7f None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:12.939785 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:33:12.945291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7527f-a145-4e8f-9d61-914e6e3ee499 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:12.945762 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7527f-a145-4e8f-9d61-914e6e3ee499 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:12.945864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7527f-a145-4e8f-9d61-914e6e3ee499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:13.011607 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-c6e7527f-a145-4e8f-9d61-914e6e3ee499 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:33:13.012810 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c6e7527f-a145-4e8f-9d61-914e6e3ee499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:13.051361 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:13.109529 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:33:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:33:13.146792 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536ae24-de20-469d-bb36-e1c1bb68ed7f None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:33:13.148364 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f536ae24-de20-469d-bb36-e1c1bb68ed7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:13.251848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536ae24-de20-469d-bb36-e1c1bb68ed7f None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:33:13.290955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:13.326696 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a23227-0637-4534-bd8e-949074c7f260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:33:13.327341 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a23227-0637-4534-bd8e-949074c7f260 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:13.327620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a23227-0637-4534-bd8e-949074c7f260 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:13.327892 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a23227-0637-4534-bd8e-949074c7f260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:13.330455 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a4a23227-0637-4534-bd8e-949074c7f260 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:13.437220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a23227-0637-4534-bd8e-949074c7f260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:33:13.439180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a23227-0637-4534-bd8e-949074c7f260 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:33:13.458409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 738/1464] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:33:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:33:15.523415 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375a11c-d728-4d7d-9d93-49f56674e958 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:15.523988 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375a11c-d728-4d7d-9d93-49f56674e958 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:15.523988 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375a11c-d728-4d7d-9d93-49f56674e958 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:15.525387 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:33:15.531086 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a5c73-89cf-4525-a04b-d04fa1dc34bc None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:15.531665 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a5c73-89cf-4525-a04b-d04fa1dc34bc None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:15.531665 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a5c73-89cf-4525-a04b-d04fa1dc34bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:15.599242 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-b03a5c73-89cf-4525-a04b-d04fa1dc34bc None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:33:15.600467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-b03a5c73-89cf-4525-a04b-d04fa1dc34bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:15.619509 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:15.699724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:33:15.738136 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375a11c-d728-4d7d-9d93-49f56674e958 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:33:15.739681 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-6375a11c-d728-4d7d-9d93-49f56674e958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:15.841166 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375a11c-d728-4d7d-9d93-49f56674e958 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:33:15.907207 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:15.914090 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426ffc3-8fa1-4ebd-96ea-03c1b2d66ef9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:33:15.914736 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426ffc3-8fa1-4ebd-96ea-03c1b2d66ef9 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:15.915019 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426ffc3-8fa1-4ebd-96ea-03c1b2d66ef9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:15.915292 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426ffc3-8fa1-4ebd-96ea-03c1b2d66ef9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:15.916923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-0426ffc3-8fa1-4ebd-96ea-03c1b2d66ef9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:16.026171 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426ffc3-8fa1-4ebd-96ea-03c1b2d66ef9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:33:16.027531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426ffc3-8fa1-4ebd-96ea-03c1b2d66ef9 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:33:16.045633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 728/1467] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:33:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:33:18.099210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7391af-6f61-48ab-bcda-991a7bc7c617 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:18.099760 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7391af-6f61-48ab-bcda-991a7bc7c617 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:18.099760 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7391af-6f61-48ab-bcda-991a7bc7c617 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:18.101170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:33:18.106561 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e94d0-6d56-4324-8aa1-c426581f3073 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:18.106981 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e94d0-6d56-4324-8aa1-c426581f3073 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:18.107122 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e94d0-6d56-4324-8aa1-c426581f3073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:18.172049 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-643e94d0-6d56-4324-8aa1-c426581f3073 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:33:18.173790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-643e94d0-6d56-4324-8aa1-c426581f3073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:18.222452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:18.270865 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:33:18.308143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7391af-6f61-48ab-bcda-991a7bc7c617 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:33:18.309767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-8f7391af-6f61-48ab-bcda-991a7bc7c617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:18.415863 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7391af-6f61-48ab-bcda-991a7bc7c617 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:33:18.465543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:18.493849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543016bb-ccd8-4c9c-9841-a1d46411dce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:33:18.494498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543016bb-ccd8-4c9c-9841-a1d46411dce9 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:18.494777 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543016bb-ccd8-4c9c-9841-a1d46411dce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:18.495050 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543016bb-ccd8-4c9c-9841-a1d46411dce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:18.496699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-543016bb-ccd8-4c9c-9841-a1d46411dce9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:18.607738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543016bb-ccd8-4c9c-9841-a1d46411dce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:33:18.609245 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543016bb-ccd8-4c9c-9841-a1d46411dce9 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:33:18.628294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 741/1470] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:33:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:33:28.268999 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=22 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:28.625413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=66 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:38.379927 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8af123-69a4-411d-8df5-45e499a16507 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:38.380526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8af123-69a4-411d-8df5-45e499a16507 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:38.380526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8af123-69a4-411d-8df5-45e499a16507 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:38.381894 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 730/1471] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:33:38.387144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f720a-3096-4f07-991c-806ba1a90731 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:38.387542 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f720a-3096-4f07-991c-806ba1a90731 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:38.387710 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f720a-3096-4f07-991c-806ba1a90731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:38.452428 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-058f720a-3096-4f07-991c-806ba1a90731 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:33:38.454234 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-058f720a-3096-4f07-991c-806ba1a90731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:38.552703 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 742/1472] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:33:38.639003 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b02d4-35a2-44f1-b659-375e1e913f93 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:38.639293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b02d4-35a2-44f1-b659-375e1e913f93 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:38.639573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b02d4-35a2-44f1-b659-375e1e913f93 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:38.640970 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 731/1473] 10.222.0.71 () {54 vars in 966 bytes} [Thu Jul 21 20:33:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:33:38.646649 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b5b38-281b-4998-a83f-04e04b862039 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:38.646939 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b5b38-281b-4998-a83f-04e04b862039 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:38.647220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b5b38-281b-4998-a83f-04e04b862039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:38.723014 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-824b5b38-281b-4998-a83f-04e04b862039 None None] MFA Rules not processed for user `ee2460e6a7e34a6881984e4f7ed294c1`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:33:38.724262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-824b5b38-281b-4998-a83f-04e04b862039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:38.747175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:38.804163 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 743/1474] 10.222.0.71 () {58 vars in 1063 bytes} [Thu Jul 21 20:33:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:33:38.811586 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b02d4-35a2-44f1-b659-375e1e913f93 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:33:38.813744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-bf6b02d4-35a2-44f1-b659-375e1e913f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:38.933314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b02d4-35a2-44f1-b659-375e1e913f93 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:33:39.018021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96c476-cd1b-43a3-b03f-47df56a2b800 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2460e6a7e34a6881984e4f7ed294c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:33:39.018686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96c476-cd1b-43a3-b03f-47df56a2b800 service glance] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:39.019018 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96c476-cd1b-43a3-b03f-47df56a2b800 service glance] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:39.019296 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96c476-cd1b-43a3-b03f-47df56a2b800 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:39.020995 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-5e96c476-cd1b-43a3-b03f-47df56a2b800 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:39.077544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:39.136775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96c476-cd1b-43a3-b03f-47df56a2b800 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:33:39.138257 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96c476-cd1b-43a3-b03f-47df56a2b800 service glance] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:33:39.158435 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 732/1475] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:33:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:33:41.341207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad1695-ec2e-488f-846b-5cc547917a14 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:41.341865 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad1695-ec2e-488f-846b-5cc547917a14 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:41.341865 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad1695-ec2e-488f-846b-5cc547917a14 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:41.343142 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 744/1476] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:33:41.349782 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b9cca-6c75-44fe-a836-69348fd498a0 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:41.350287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b9cca-6c75-44fe-a836-69348fd498a0 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:41.350393 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b9cca-6c75-44fe-a836-69348fd498a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:41.417493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-698b9cca-6c75-44fe-a836-69348fd498a0 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:33:41.418686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-698b9cca-6c75-44fe-a836-69348fd498a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:41.522154 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 733/1477] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:33:41.544918 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad1695-ec2e-488f-846b-5cc547917a14 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:33:41.546498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-02ad1695-ec2e-488f-846b-5cc547917a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:41.675624 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad1695-ec2e-488f-846b-5cc547917a14 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:33:41.726748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:41.773443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de791e-bdc9-493f-9467-411b2fe28f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:33:41.774146 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de791e-bdc9-493f-9467-411b2fe28f2c admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:41.774422 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de791e-bdc9-493f-9467-411b2fe28f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:41.774694 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de791e-bdc9-493f-9467-411b2fe28f2c admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:41.775471 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de791e-bdc9-493f-9467-411b2fe28f2c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:33:41.777293 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: WARNING py.warnings [None req-13de791e-bdc9-493f-9467-411b2fe28f2c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:33:41.777293 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: warnings.warn(msg) Jul 21 20:33:41.777293 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: Jul 21 20:33:41.778446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de791e-bdc9-493f-9467-411b2fe28f2c admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:33:41.798311 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 745/1478] 10.222.0.71 () {56 vars in 1108 bytes} [Thu Jul 21 20:33:41 2022] GET /identity/v3/projects => generated 2553 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:33:43.915618 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dca6b1-661e-4742-a406-67a4531ee52d None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:43.916157 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dca6b1-661e-4742-a406-67a4531ee52d None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:43.916157 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dca6b1-661e-4742-a406-67a4531ee52d None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:43.917591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 734/1479] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:33:43.923375 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8dd6c-ce67-4a1c-b65b-be9de0dc89bc None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:43.923855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8dd6c-ce67-4a1c-b65b-be9de0dc89bc None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:43.923968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8dd6c-ce67-4a1c-b65b-be9de0dc89bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:43.991969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-6ec8dd6c-ce67-4a1c-b65b-be9de0dc89bc None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:33:43.993213 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-6ec8dd6c-ce67-4a1c-b65b-be9de0dc89bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:44.095040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 746/1480] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:33:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:33:44.138471 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dca6b1-661e-4742-a406-67a4531ee52d None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:33:44.140717 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-f7dca6b1-661e-4742-a406-67a4531ee52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:44.152520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:44.409109 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dca6b1-661e-4742-a406-67a4531ee52d None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:33:44.499047 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:44.534603 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809d80c-b7d8-4f6b-b48d-59817ff60fbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:33:44.535307 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809d80c-b7d8-4f6b-b48d-59817ff60fbc service nova] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:44.535572 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809d80c-b7d8-4f6b-b48d-59817ff60fbc service nova] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:44.535845 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809d80c-b7d8-4f6b-b48d-59817ff60fbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:44.537586 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-2809d80c-b7d8-4f6b-b48d-59817ff60fbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:44.632596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809d80c-b7d8-4f6b-b48d-59817ff60fbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:33:44.634163 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809d80c-b7d8-4f6b-b48d-59817ff60fbc service nova] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:33:44.653422 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 735/1481] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:33:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:33:46.960814 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577f24e-491f-4cd5-aab8-278802bc19bd None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:46.961450 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577f24e-491f-4cd5-aab8-278802bc19bd None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:46.961450 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577f24e-491f-4cd5-aab8-278802bc19bd None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:46.962824 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 747/1482] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:33:46.969095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11214f-4830-460f-af57-9d72a393d539 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:46.969928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11214f-4830-460f-af57-9d72a393d539 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:46.969928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11214f-4830-460f-af57-9d72a393d539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:47.040958 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-7d11214f-4830-460f-af57-9d72a393d539 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:33:47.042099 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-7d11214f-4830-460f-af57-9d72a393d539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:47.090608 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:47.144010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 736/1483] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:33:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:33:47.181387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577f24e-491f-4cd5-aab8-278802bc19bd None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:33:47.182988 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3577f24e-491f-4cd5-aab8-278802bc19bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:47.214910 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:47.294575 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577f24e-491f-4cd5-aab8-278802bc19bd None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:33:47.371189 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bdfd1-9467-4bfb-b96a-0268ba3de6ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:33:47.371839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bdfd1-9467-4bfb-b96a-0268ba3de6ae service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:47.372564 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bdfd1-9467-4bfb-b96a-0268ba3de6ae service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:47.372564 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bdfd1-9467-4bfb-b96a-0268ba3de6ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:47.374107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-c23bdfd1-9467-4bfb-b96a-0268ba3de6ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:47.451256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:47.490229 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bdfd1-9467-4bfb-b96a-0268ba3de6ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:33:47.491804 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bdfd1-9467-4bfb-b96a-0268ba3de6ae service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:33:47.511310 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 748/1484] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:33:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:33:49.692575 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5d207-42eb-40d3-a5e4-69bd07142fc9 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:49.693204 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5d207-42eb-40d3-a5e4-69bd07142fc9 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:49.693204 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5d207-42eb-40d3-a5e4-69bd07142fc9 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:49.694582 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 737/1485] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:33:49.707015 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa2c22-85ae-4655-a3eb-c29bbf88c0d0 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:49.707558 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa2c22-85ae-4655-a3eb-c29bbf88c0d0 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:49.707558 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa2c22-85ae-4655-a3eb-c29bbf88c0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:49.784346 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-8caa2c22-85ae-4655-a3eb-c29bbf88c0d0 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:33:49.785626 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-8caa2c22-85ae-4655-a3eb-c29bbf88c0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:49.893637 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 749/1486] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:33:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:33:49.931944 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5d207-42eb-40d3-a5e4-69bd07142fc9 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:33:49.933606 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-74a5d207-42eb-40d3-a5e4-69bd07142fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:50.057920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5d207-42eb-40d3-a5e4-69bd07142fc9 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:33:50.176742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:50.180612 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9255da4-dd9d-499e-8b5f-eeca36b3c22f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:33:50.181810 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9255da4-dd9d-499e-8b5f-eeca36b3c22f service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:50.182498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9255da4-dd9d-499e-8b5f-eeca36b3c22f service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:50.183013 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9255da4-dd9d-499e-8b5f-eeca36b3c22f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:50.186345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c9255da4-dd9d-499e-8b5f-eeca36b3c22f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:50.306651 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9255da4-dd9d-499e-8b5f-eeca36b3c22f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:33:50.308168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9255da4-dd9d-499e-8b5f-eeca36b3c22f service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:33:50.327151 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 738/1487] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:33:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:33:52.727861 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8cc65-8790-45f8-8fed-6818ece47959 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:52.728432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8cc65-8790-45f8-8fed-6818ece47959 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:52.728432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8cc65-8790-45f8-8fed-6818ece47959 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:52.729744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 750/1488] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:33:52.736387 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6154b-f2ef-4b51-ac56-c61ac617d22f None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:52.736910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6154b-f2ef-4b51-ac56-c61ac617d22f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:52.736910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6154b-f2ef-4b51-ac56-c61ac617d22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:52.810723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-51d6154b-f2ef-4b51-ac56-c61ac617d22f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:33:52.812043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-51d6154b-f2ef-4b51-ac56-c61ac617d22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:52.876846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:52.931890 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 739/1489] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:33:52.974882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8cc65-8790-45f8-8fed-6818ece47959 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:33:52.975621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-d6f8cc65-8790-45f8-8fed-6818ece47959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:53.015260 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:53.093178 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8cc65-8790-45f8-8fed-6818ece47959 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:33:53.184475 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c009c94-a1b5-4c2a-ac5b-40f6f2ae378c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:33:53.185143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c009c94-a1b5-4c2a-ac5b-40f6f2ae378c service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:53.185427 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c009c94-a1b5-4c2a-ac5b-40f6f2ae378c service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:53.185699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c009c94-a1b5-4c2a-ac5b-40f6f2ae378c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:53.187402 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-7c009c94-a1b5-4c2a-ac5b-40f6f2ae378c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:53.275771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:53.325454 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c009c94-a1b5-4c2a-ac5b-40f6f2ae378c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:33:53.328266 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c009c94-a1b5-4c2a-ac5b-40f6f2ae378c service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:33:53.366434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 751/1490] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:33:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:33:55.749252 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa9541-b13f-4c90-bc64-f55892889b51 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:55.749770 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa9541-b13f-4c90-bc64-f55892889b51 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:55.750683 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa9541-b13f-4c90-bc64-f55892889b51 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:55.752484 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 740/1491] 10.222.0.71 () {54 vars in 926 bytes} [Thu Jul 21 20:33:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:33:55.759537 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cc432-32ec-4fc8-92ce-dcee82e66399 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:55.760159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cc432-32ec-4fc8-92ce-dcee82e66399 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:55.760159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cc432-32ec-4fc8-92ce-dcee82e66399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:55.854370 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-9d5cc432-32ec-4fc8-92ce-dcee82e66399 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:33:55.855627 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-9d5cc432-32ec-4fc8-92ce-dcee82e66399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:55.987697 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 752/1492] 10.222.0.71 () {58 vars in 1023 bytes} [Thu Jul 21 20:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:33:56.037112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa9541-b13f-4c90-bc64-f55892889b51 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:33:56.039910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-4baa9541-b13f-4c90-bc64-f55892889b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:56.164113 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa9541-b13f-4c90-bc64-f55892889b51 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:33:56.277740 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306b523-0592-475b-a2ae-fa7f28e088c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:33:56.278392 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306b523-0592-475b-a2ae-fa7f28e088c8 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:33:56.278668 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306b523-0592-475b-a2ae-fa7f28e088c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:33:56.278934 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306b523-0592-475b-a2ae-fa7f28e088c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:33:56.280695 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-e306b523-0592-475b-a2ae-fa7f28e088c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:33:56.291727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:33:56.406091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e306b523-0592-475b-a2ae-fa7f28e088c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:33:56.408456 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e306b523-0592-475b-a2ae-fa7f28e088c8 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:33:56.431663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 741/1493] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:33:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:34:05.983888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=82 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:34:06.428724 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=50 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:35:24.029720 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512babc2-8e5f-4e87-bbfb-4e56a0762ed1 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:35:24.030353 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512babc2-8e5f-4e87-bbfb-4e56a0762ed1 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:35:24.030353 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512babc2-8e5f-4e87-bbfb-4e56a0762ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:35:24.107081 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-512babc2-8e5f-4e87-bbfb-4e56a0762ed1 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:35:24.108832 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-512babc2-8e5f-4e87-bbfb-4e56a0762ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:35:24.218152 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 753/1494] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:35:24.227098 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e968f0c-46bb-4fed-9db0-e832f8bc917f None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:35:24.227668 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e968f0c-46bb-4fed-9db0-e832f8bc917f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:35:24.227668 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e968f0c-46bb-4fed-9db0-e832f8bc917f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:35:24.306950 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-8e968f0c-46bb-4fed-9db0-e832f8bc917f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:35:24.308090 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-8e968f0c-46bb-4fed-9db0-e832f8bc917f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:35:24.407008 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 742/1495] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:35:24.411818 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512babc2-8e5f-4e87-bbfb-4e56a0762ed1 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:35:24.413042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-512babc2-8e5f-4e87-bbfb-4e56a0762ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:35:24.425251 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512babc2-8e5f-4e87-bbfb-4e56a0762ed1 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:35:24.490435 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:35:24.525495 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dafdc0-b6b3-46ef-9c50-588ae1288a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:35:24.526157 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dafdc0-b6b3-46ef-9c50-588ae1288a5e admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:35:24.526441 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dafdc0-b6b3-46ef-9c50-588ae1288a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:35:24.526698 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dafdc0-b6b3-46ef-9c50-588ae1288a5e admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:35:24.527435 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dafdc0-b6b3-46ef-9c50-588ae1288a5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:35:24.527737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dafdc0-b6b3-46ef-9c50-588ae1288a5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:35:24.528810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: WARNING py.warnings [None req-32dafdc0-b6b3-46ef-9c50-588ae1288a5e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:35:24.528810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: warnings.warn(msg) Jul 21 20:35:24.528810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: Jul 21 20:35:24.529753 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dafdc0-b6b3-46ef-9c50-588ae1288a5e admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:35:24.542168 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 754/1496] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:35:24.551709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e968f0c-46bb-4fed-9db0-e832f8bc917f None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:35:24.561376 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e968f0c-46bb-4fed-9db0-e832f8bc917f None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:35:24.626117 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:35:24.667235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43ae00-7643-445d-9e12-46ae2461e807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:35:24.667907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43ae00-7643-445d-9e12-46ae2461e807 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:35:24.668185 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43ae00-7643-445d-9e12-46ae2461e807 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:35:24.668456 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43ae00-7643-445d-9e12-46ae2461e807 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:35:24.669209 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf43ae00-7643-445d-9e12-46ae2461e807 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:35:24.671647 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: WARNING py.warnings [None req-cf43ae00-7643-445d-9e12-46ae2461e807 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:35:24.671647 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: warnings.warn(msg) Jul 21 20:35:24.671647 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: Jul 21 20:35:24.672889 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf43ae00-7643-445d-9e12-46ae2461e807 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:35:24.738479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 743/1497] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:35:24 2022] POST /identity/v3/projects => generated 379 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:35:24.743099 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dafdc0-b6b3-46ef-9c50-588ae1288a5e admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:35:24.744258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-32dafdc0-b6b3-46ef-9c50-588ae1288a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:35:24.777727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dafdc0-b6b3-46ef-9c50-588ae1288a5e admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:35:24.875918 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d2587-2eb4-42e5-a0b3-3f4b40c6e5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:35:24.876494 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d2587-2eb4-42e5-a0b3-3f4b40c6e5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:35:24.876769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d2587-2eb4-42e5-a0b3-3f4b40c6e5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:35:24.877037 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d2587-2eb4-42e5-a0b3-3f4b40c6e5a7 admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:35:24.877750 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d2587-2eb4-42e5-a0b3-3f4b40c6e5a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:35:24.878778 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: WARNING py.warnings [None req-6e3d2587-2eb4-42e5-a0b3-3f4b40c6e5a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:35:24.878778 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: warnings.warn(msg) Jul 21 20:35:24.878778 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: Jul 21 20:35:24.880004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d2587-2eb4-42e5-a0b3-3f4b40c6e5a7 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:35:24.925329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 755/1498] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:35:24 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:35:24.929656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43ae00-7643-445d-9e12-46ae2461e807 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:35:24.931194 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-cf43ae00-7643-445d-9e12-46ae2461e807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:35:24.996226 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43ae00-7643-445d-9e12-46ae2461e807 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:35:25.081434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:35:25.093659 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be026bba-13b1-4457-a341-9bfcb9797dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:35:25.094338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be026bba-13b1-4457-a341-9bfcb9797dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:35:25.094686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be026bba-13b1-4457-a341-9bfcb9797dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:35:25.094967 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be026bba-13b1-4457-a341-9bfcb9797dc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:35:25.096002 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be026bba-13b1-4457-a341-9bfcb9797dc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:35:25.097281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: WARNING py.warnings [None req-be026bba-13b1-4457-a341-9bfcb9797dc6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:35:25.097281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: warnings.warn(msg) Jul 21 20:35:25.097281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: Jul 21 20:35:25.098179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be026bba-13b1-4457-a341-9bfcb9797dc6 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:35:25.112172 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 744/1499] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:35:25.117953 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d2587-2eb4-42e5-a0b3-3f4b40c6e5a7 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:35:25.119612 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-6e3d2587-2eb4-42e5-a0b3-3f4b40c6e5a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:35:25.131168 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d2587-2eb4-42e5-a0b3-3f4b40c6e5a7 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:35:25.156478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:35:25.226358 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780285cc-440e-49c3-9ccc-682b3cf033ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:35:25.227025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780285cc-440e-49c3-9ccc-682b3cf033ff admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:35:25.227309 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780285cc-440e-49c3-9ccc-682b3cf033ff admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:35:25.227582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780285cc-440e-49c3-9ccc-682b3cf033ff admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:35:25.228308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780285cc-440e-49c3-9ccc-682b3cf033ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:35:25.229358 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: WARNING py.warnings [None req-780285cc-440e-49c3-9ccc-682b3cf033ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:35:25.229358 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: warnings.warn(msg) Jul 21 20:35:25.229358 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: Jul 21 20:35:25.230242 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780285cc-440e-49c3-9ccc-682b3cf033ff admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:35:25.245643 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 756/1500] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:35:25.250441 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be026bba-13b1-4457-a341-9bfcb9797dc6 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:35:25.252076 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-be026bba-13b1-4457-a341-9bfcb9797dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:35:25.285336 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be026bba-13b1-4457-a341-9bfcb9797dc6 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:35:25.378229 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a2381-5b42-4908-b44d-dcc475cd17c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:35:25.379113 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a2381-5b42-4908-b44d-dcc475cd17c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:35:25.379394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a2381-5b42-4908-b44d-dcc475cd17c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:35:25.379666 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a2381-5b42-4908-b44d-dcc475cd17c7 admin admin] PATH_INFO: `/v3/projects/5acb9a37595f4c95a91d1aaa6c4da37f/users/4c5f1c5dd76245208518ec987e8f112b/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:35:25.402351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a2381-5b42-4908-b44d-dcc475cd17c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5acb9a37595f4c95a91d1aaa6c4da37f, user_id=4c5f1c5dd76245208518ec987e8f112b, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:35:25.403319 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: WARNING py.warnings [None req-a65a2381-5b42-4908-b44d-dcc475cd17c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:35:25.403319 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: warnings.warn(msg) Jul 21 20:35:25.403319 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: Jul 21 20:35:25.405314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a2381-5b42-4908-b44d-dcc475cd17c7 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:35:25.429201 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 745/1501] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:35:25 2022] PUT /identity/v3/projects/5acb9a37595f4c95a91d1aaa6c4da37f/users/4c5f1c5dd76245208518ec987e8f112b/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:35:25.441800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867458a-fc00-4634-a93c-34897c9ec703 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:35:25.442097 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867458a-fc00-4634-a93c-34897c9ec703 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:35:25.442399 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867458a-fc00-4634-a93c-34897c9ec703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:35:25.500034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-d867458a-fc00-4634-a93c-34897c9ec703 None None] MFA Rules not processed for user `4c5f1c5dd76245208518ec987e8f112b`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:35:25.501298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-d867458a-fc00-4634-a93c-34897c9ec703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:35:25.578629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 757/1502] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:35:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:35:25.591889 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c2112-2ce8-4bb2-a709-a5d544ded676 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:35:25.592182 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c2112-2ce8-4bb2-a709-a5d544ded676 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:35:25.592463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c2112-2ce8-4bb2-a709-a5d544ded676 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:35:25.594155 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 746/1503] 10.222.0.71 () {54 vars in 964 bytes} [Thu Jul 21 20:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:35:25.599186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db567d-283b-4a66-9471-b34f36b9105c None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:35:25.599469 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db567d-283b-4a66-9471-b34f36b9105c None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:35:25.599744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db567d-283b-4a66-9471-b34f36b9105c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:35:25.629800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:35:25.674707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-51db567d-283b-4a66-9471-b34f36b9105c None None] MFA Rules not processed for user `34db429cd31549f78daf5b1cbf6f0bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:35:25.675925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-51db567d-283b-4a66-9471-b34f36b9105c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:35:25.752049 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 758/1504] 10.222.0.71 () {58 vars in 1061 bytes} [Thu Jul 21 20:35:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:35:25.759741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c2112-2ce8-4bb2-a709-a5d544ded676 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:35:25.761317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-fe1c2112-2ce8-4bb2-a709-a5d544ded676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:35:25.810355 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:35:25.865253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c2112-2ce8-4bb2-a709-a5d544ded676 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:35:25.938644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4f982-c2a4-47c9-bd13-4aaf08bb3227 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:35:25.939216 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4f982-c2a4-47c9-bd13-4aaf08bb3227 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:35:25.939497 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4f982-c2a4-47c9-bd13-4aaf08bb3227 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:35:25.939774 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4f982-c2a4-47c9-bd13-4aaf08bb3227 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:35:25.941474 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-38c4f982-c2a4-47c9-bd13-4aaf08bb3227 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:35:26.040587 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4f982-c2a4-47c9-bd13-4aaf08bb3227 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:35:26.042059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4f982-c2a4-47c9-bd13-4aaf08bb3227 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:35:26.046797 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:35:26.069745 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 747/1505] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:35:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:35:26.115231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bc9ea-4402-4845-9878-e9a4e0ef9dc9 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:35:26.115519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bc9ea-4402-4845-9878-e9a4e0ef9dc9 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:35:26.115799 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bc9ea-4402-4845-9878-e9a4e0ef9dc9 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:35:26.117131 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 759/1506] 10.222.0.71 () {52 vars in 829 bytes} [Thu Jul 21 20:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:35:26.120720 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4f982-c2a4-47c9-bd13-4aaf08bb3227 service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:35:26.122109 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-38c4f982-c2a4-47c9-bd13-4aaf08bb3227 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:35:26.222237 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4f982-c2a4-47c9-bd13-4aaf08bb3227 service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:35:26.327698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea570f28-625a-4b4f-ada9-adf407bc84a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:35:26.328573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea570f28-625a-4b4f-ada9-adf407bc84a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:35:26.328854 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea570f28-625a-4b4f-ada9-adf407bc84a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:35:26.329158 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea570f28-625a-4b4f-ada9-adf407bc84a6 admin admin] PATH_INFO: `/v3/users/4c5f1c5dd76245208518ec987e8f112b` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:35:26.341525 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea570f28-625a-4b4f-ada9-adf407bc84a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c5f1c5dd76245208518ec987e8f112b)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:35:26.343077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: WARNING py.warnings [None req-ea570f28-625a-4b4f-ada9-adf407bc84a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:35:26.343077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: warnings.warn(msg) Jul 21 20:35:26.343077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: Jul 21 20:35:26.344554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea570f28-625a-4b4f-ada9-adf407bc84a6 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:35:26.354229 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:35:26.407643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-ea570f28-625a-4b4f-ada9-adf407bc84a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c5f1c5dd76245208518ec987e8f112b'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:35:26.417484 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-ea570f28-625a-4b4f-ada9-adf407bc84a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c5f1c5dd76245208518ec987e8f112b'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:35:26.417905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-ea570f28-625a-4b4f-ada9-adf407bc84a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c5f1c5dd76245208518ec987e8f112b'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:35:26.436428 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-ea570f28-625a-4b4f-ada9-adf407bc84a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c5f1c5dd76245208518ec987e8f112b'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:35:26.448160 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-ea570f28-625a-4b4f-ada9-adf407bc84a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c5f1c5dd76245208518ec987e8f112b'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:35:26.454778 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 748/1507] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:35:26 2022] DELETE /identity/v3/users/4c5f1c5dd76245208518ec987e8f112b => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:35:26.459427 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a95483-c77b-4156-9762-0aa41278705c None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:35:26.459837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a95483-c77b-4156-9762-0aa41278705c None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:35:26.459984 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a95483-c77b-4156-9762-0aa41278705c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:35:26.517091 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-b3a95483-c77b-4156-9762-0aa41278705c None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:35:26.518313 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-b3a95483-c77b-4156-9762-0aa41278705c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:35:26.594086 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:35:26.607675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 760/1508] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:35:26.617982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea570f28-625a-4b4f-ada9-adf407bc84a6 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:35:26.619289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-ea570f28-625a-4b4f-ada9-adf407bc84a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:35:26.719788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea570f28-625a-4b4f-ada9-adf407bc84a6 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:35:26.779958 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:35:26.801260 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4d299-245c-497f-be72-2d472f75db30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:35:26.801931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4d299-245c-497f-be72-2d472f75db30 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:35:26.802189 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4d299-245c-497f-be72-2d472f75db30 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:35:26.802460 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4d299-245c-497f-be72-2d472f75db30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:35:26.804100 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c6e4d299-245c-497f-be72-2d472f75db30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:35:26.912817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e4d299-245c-497f-be72-2d472f75db30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:35:26.914310 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e4d299-245c-497f-be72-2d472f75db30 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:35:26.932920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 749/1509] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:35:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:35:27.270517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a95483-c77b-4156-9762-0aa41278705c None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:35:27.272688 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-b3a95483-c77b-4156-9762-0aa41278705c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:35:27.294106 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a95483-c77b-4156-9762-0aa41278705c None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:35:27.400684 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242d525-227a-4c01-80e7-130952948452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:35:27.401647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242d525-227a-4c01-80e7-130952948452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:35:27.401933 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242d525-227a-4c01-80e7-130952948452 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:35:27.402206 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242d525-227a-4c01-80e7-130952948452 admin admin] PATH_INFO: `/v3/projects/5acb9a37595f4c95a91d1aaa6c4da37f` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:35:27.412675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242d525-227a-4c01-80e7-130952948452 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5acb9a37595f4c95a91d1aaa6c4da37f)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:35:27.413744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: WARNING py.warnings [None req-a242d525-227a-4c01-80e7-130952948452 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:35:27.413744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: warnings.warn(msg) Jul 21 20:35:27.413744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: Jul 21 20:35:27.414975 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242d525-227a-4c01-80e7-130952948452 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:35:27.472701 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-a242d525-227a-4c01-80e7-130952948452 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5acb9a37595f4c95a91d1aaa6c4da37f', 'request_id': 'req-a242d525-227a-4c01-80e7-130952948452'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:35:27.482485 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-a242d525-227a-4c01-80e7-130952948452 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5acb9a37595f4c95a91d1aaa6c4da37f', 'request_id': 'req-a242d525-227a-4c01-80e7-130952948452'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:35:27.492578 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-a242d525-227a-4c01-80e7-130952948452 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5acb9a37595f4c95a91d1aaa6c4da37f', 'request_id': 'req-a242d525-227a-4c01-80e7-130952948452'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:35:27.522329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-a242d525-227a-4c01-80e7-130952948452 admin admin] The token cache is being invalidate because project 5acb9a37595f4c95a91d1aaa6c4da37f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:35:27.522619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-a242d525-227a-4c01-80e7-130952948452 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:35:27.523395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-a242d525-227a-4c01-80e7-130952948452 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:35:27.524881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 761/1510] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:35:27 2022] DELETE /identity/v3/projects/5acb9a37595f4c95a91d1aaa6c4da37f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:35:36.930441 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=62 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:35:37.520288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:08.727716 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fe0e4-5a5e-4ed4-b101-2b260fa58d6f None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:08.728320 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fe0e4-5a5e-4ed4-b101-2b260fa58d6f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:08.728320 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fe0e4-5a5e-4ed4-b101-2b260fa58d6f None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:08.729731 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:08.734801 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27115e27-45b5-4f6f-a561-46c34e5fafe0 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:08.735254 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27115e27-45b5-4f6f-a561-46c34e5fafe0 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:08.735422 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27115e27-45b5-4f6f-a561-46c34e5fafe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:08.814224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-27115e27-45b5-4f6f-a561-46c34e5fafe0 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:08.815439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-27115e27-45b5-4f6f-a561-46c34e5fafe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:08.920501 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:37:09.073965 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fe0e4-5a5e-4ed4-b101-2b260fa58d6f None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:09.076078 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-925fe0e4-5a5e-4ed4-b101-2b260fa58d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:09.193405 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fe0e4-5a5e-4ed4-b101-2b260fa58d6f None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:09.198921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9c043-886f-41ef-9f98-d9c2f90bb4c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:09.199560 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9c043-886f-41ef-9f98-d9c2f90bb4c1 service nova] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:09.199845 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9c043-886f-41ef-9f98-d9c2f90bb4c1 service nova] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:09.200119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9c043-886f-41ef-9f98-d9c2f90bb4c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:09.208316 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9c043-886f-41ef-9f98-d9c2f90bb4c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:09.209936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9c043-886f-41ef-9f98-d9c2f90bb4c1 service nova] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:09.219572 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 751/1513] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:37:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:37:10.058457 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3b7f9-6afc-471d-8369-9616ab9c100a None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:10.058981 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3b7f9-6afc-471d-8369-9616ab9c100a None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:10.059088 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3b7f9-6afc-471d-8369-9616ab9c100a None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:10.060559 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:10.065758 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c047fc2-a3a1-488f-82bc-9650259e15b2 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:10.066199 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c047fc2-a3a1-488f-82bc-9650259e15b2 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:10.066302 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c047fc2-a3a1-488f-82bc-9650259e15b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:10.084960 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-1c047fc2-a3a1-488f-82bc-9650259e15b2 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:10.086194 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-1c047fc2-a3a1-488f-82bc-9650259e15b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:10.094128 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:37:10.248130 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3b7f9-6afc-471d-8369-9616ab9c100a None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:10.262409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3b7f9-6afc-471d-8369-9616ab9c100a None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:10.267826 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262ad1e-3d8d-470b-8198-867f407fc406 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:10.268459 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262ad1e-3d8d-470b-8198-867f407fc406 service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:10.268777 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262ad1e-3d8d-470b-8198-867f407fc406 service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:10.269058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262ad1e-3d8d-470b-8198-867f407fc406 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:10.278020 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262ad1e-3d8d-470b-8198-867f407fc406 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:10.279632 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262ad1e-3d8d-470b-8198-867f407fc406 service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:10.289096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 764/1516] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:37:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:37:11.399715 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa8afd-a92e-49eb-9464-0d038d6f6daa None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:11.400287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa8afd-a92e-49eb-9464-0d038d6f6daa None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:11.400287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa8afd-a92e-49eb-9464-0d038d6f6daa None None] PATH_INFO: `/v3` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:11.401611 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 753/1517] 10.222.0.79 () {54 vars in 925 bytes} [Thu Jul 21 20:37:11 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:11.408399 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c83363-bf0d-4b4f-9fc1-a9d5d3d3b18e None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:11.408827 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c83363-bf0d-4b4f-9fc1-a9d5d3d3b18e None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:11.408958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c83363-bf0d-4b4f-9fc1-a9d5d3d3b18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:11.468252 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-11c83363-bf0d-4b4f-9fc1-a9d5d3d3b18e None None] MFA Rules not processed for user `364e80ec79b94812a1df76c62c2b13a0`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:11.469793 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-11c83363-bf0d-4b4f-9fc1-a9d5d3d3b18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:11.487358 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:11.663382 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 765/1518] 10.222.0.79 () {58 vars in 1015 bytes} [Thu Jul 21 20:37:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:37:15.493399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1294c-4023-48ae-884b-605ae3645ad9 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:15.493908 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1294c-4023-48ae-884b-605ae3645ad9 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:15.493908 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1294c-4023-48ae-884b-605ae3645ad9 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:15.495264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 754/1519] 10.222.0.71 () {54 vars in 961 bytes} [Thu Jul 21 20:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:15.500216 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10a497-2ea7-4d68-a4d9-66795213c806 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:15.500663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10a497-2ea7-4d68-a4d9-66795213c806 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:15.500775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10a497-2ea7-4d68-a4d9-66795213c806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:15.518512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-4a10a497-2ea7-4d68-a4d9-66795213c806 None None] MFA Rules not processed for user `364e80ec79b94812a1df76c62c2b13a0`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:15.519719 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-4a10a497-2ea7-4d68-a4d9-66795213c806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:15.528990 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 766/1520] 10.222.0.71 () {58 vars in 1058 bytes} [Thu Jul 21 20:37:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:37:15.536327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1294c-4023-48ae-884b-605ae3645ad9 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:15.550033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1294c-4023-48ae-884b-605ae3645ad9 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:15.554911 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efe80b-19ff-4441-97d3-8fe87bb4fc41 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '364e80ec79b94812a1df76c62c2b13a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:15.555558 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efe80b-19ff-4441-97d3-8fe87bb4fc41 service placement] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:15.555839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efe80b-19ff-4441-97d3-8fe87bb4fc41 service placement] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:15.556137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efe80b-19ff-4441-97d3-8fe87bb4fc41 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:15.564370 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20efe80b-19ff-4441-97d3-8fe87bb4fc41 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:15.565950 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20efe80b-19ff-4441-97d3-8fe87bb4fc41 service placement] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:15.575456 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 755/1521] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:37:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:37:21.535566 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2295d1-184c-4078-8fd1-9c7faf210174 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:21.536163 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2295d1-184c-4078-8fd1-9c7faf210174 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:21.536163 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2295d1-184c-4078-8fd1-9c7faf210174 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:21.537532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:21.543014 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764faac-1e26-4c93-b06b-7202c1245b61 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:21.543461 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764faac-1e26-4c93-b06b-7202c1245b61 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:21.543571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764faac-1e26-4c93-b06b-7202c1245b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:21.561263 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-1764faac-1e26-4c93-b06b-7202c1245b61 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:21.562449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-1764faac-1e26-4c93-b06b-7202c1245b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:21.570108 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:37:21.733683 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2295d1-184c-4078-8fd1-9c7faf210174 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:21.748304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2295d1-184c-4078-8fd1-9c7faf210174 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:21.753736 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d935b54-6247-4581-ae41-a1a79b67e658 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:21.754359 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d935b54-6247-4581-ae41-a1a79b67e658 service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:21.754677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d935b54-6247-4581-ae41-a1a79b67e658 service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:21.754991 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d935b54-6247-4581-ae41-a1a79b67e658 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:21.763772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d935b54-6247-4581-ae41-a1a79b67e658 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:21.765512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d935b54-6247-4581-ae41-a1a79b67e658 service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:21.774855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 768/1524] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:37:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:37:22.601526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6dc9d-d228-45c1-868b-f0fd8efe6276 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:22.602049 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6dc9d-d228-45c1-868b-f0fd8efe6276 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:22.602173 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6dc9d-d228-45c1-868b-f0fd8efe6276 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:22.603455 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:22.608924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5114286-e4e7-4517-b4fa-7acdf3582866 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:22.609367 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5114286-e4e7-4517-b4fa-7acdf3582866 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:22.609515 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5114286-e4e7-4517-b4fa-7acdf3582866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:22.627585 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-f5114286-e4e7-4517-b4fa-7acdf3582866 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:22.628787 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f5114286-e4e7-4517-b4fa-7acdf3582866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:22.636688 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:37:22.788716 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6dc9d-d228-45c1-868b-f0fd8efe6276 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:22.802972 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6dc9d-d228-45c1-868b-f0fd8efe6276 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:22.808899 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c297b8-9840-4c81-8e37-6fd02d007544 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:22.809561 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c297b8-9840-4c81-8e37-6fd02d007544 service nova] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:22.809845 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c297b8-9840-4c81-8e37-6fd02d007544 service nova] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:22.810118 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c297b8-9840-4c81-8e37-6fd02d007544 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:22.821949 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c297b8-9840-4c81-8e37-6fd02d007544 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:22.824114 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c297b8-9840-4c81-8e37-6fd02d007544 service nova] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:22.834011 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 758/1527] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:37:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:37:27.097399 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6c88c-a585-44a5-9c66-bec1e53676a7 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:27.098018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6c88c-a585-44a5-9c66-bec1e53676a7 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:27.098018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6c88c-a585-44a5-9c66-bec1e53676a7 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:27.099487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:27.106281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79417f-fa61-44b1-b7c9-4d1b0cb1c7a8 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:27.106794 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79417f-fa61-44b1-b7c9-4d1b0cb1c7a8 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:27.106901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79417f-fa61-44b1-b7c9-4d1b0cb1c7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:27.127481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-9e79417f-fa61-44b1-b7c9-4d1b0cb1c7a8 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:27.128737 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-9e79417f-fa61-44b1-b7c9-4d1b0cb1c7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:27.137832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:37:27.216155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6c88c-a585-44a5-9c66-bec1e53676a7 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:27.217997 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-34e6c88c-a585-44a5-9c66-bec1e53676a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:27.303001 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6c88c-a585-44a5-9c66-bec1e53676a7 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:27.306677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291c7a8-3dba-4b70-9ded-de77f13cabd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:27.307366 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291c7a8-3dba-4b70-9ded-de77f13cabd9 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:27.307664 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291c7a8-3dba-4b70-9ded-de77f13cabd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:27.307941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291c7a8-3dba-4b70-9ded-de77f13cabd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:27.317441 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291c7a8-3dba-4b70-9ded-de77f13cabd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:27.318933 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291c7a8-3dba-4b70-9ded-de77f13cabd9 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:27.328678 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 771/1530] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:37:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:37:29.382717 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e5eaa-8051-48e0-9b4c-11f6e5324133 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:29.383312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e5eaa-8051-48e0-9b4c-11f6e5324133 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:29.383312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e5eaa-8051-48e0-9b4c-11f6e5324133 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:29.384650 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:29.390026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d15b2-6493-4fc8-ada4-cd596b4be40f None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:29.390444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d15b2-6493-4fc8-ada4-cd596b4be40f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:29.390586 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d15b2-6493-4fc8-ada4-cd596b4be40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:29.410279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-5e0d15b2-6493-4fc8-ada4-cd596b4be40f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:29.411502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-5e0d15b2-6493-4fc8-ada4-cd596b4be40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:29.419974 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:37:29.456559 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e5eaa-8051-48e0-9b4c-11f6e5324133 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:29.468155 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e5eaa-8051-48e0-9b4c-11f6e5324133 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:29.471960 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83598166-bc68-4d9d-a91f-f69c2d99006c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:29.472606 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83598166-bc68-4d9d-a91f-f69c2d99006c service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:29.472996 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83598166-bc68-4d9d-a91f-f69c2d99006c service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:29.473307 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83598166-bc68-4d9d-a91f-f69c2d99006c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:29.485185 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83598166-bc68-4d9d-a91f-f69c2d99006c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:29.486623 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83598166-bc68-4d9d-a91f-f69c2d99006c service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:29.496189 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 761/1533] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:37:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:37:33.425218 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec238b-fe7c-499d-b35a-1a4a0c7e3945 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:33.425790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec238b-fe7c-499d-b35a-1a4a0c7e3945 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:33.425790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec238b-fe7c-499d-b35a-1a4a0c7e3945 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:33.427116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:33.433122 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fcb2d-c658-455b-b83f-70af14b6bbb4 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:33.433565 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fcb2d-c658-455b-b83f-70af14b6bbb4 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:33.433714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fcb2d-c658-455b-b83f-70af14b6bbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:33.452418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-338fcb2d-c658-455b-b83f-70af14b6bbb4 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:33.453662 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-338fcb2d-c658-455b-b83f-70af14b6bbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:33.461603 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:37:33.767149 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec238b-fe7c-499d-b35a-1a4a0c7e3945 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:33.768830 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-7eec238b-fe7c-499d-b35a-1a4a0c7e3945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:33.833921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:33.848878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec238b-fe7c-499d-b35a-1a4a0c7e3945 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:33.852443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fb772-288a-4a88-be25-489541d7afba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:33.853112 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fb772-288a-4a88-be25-489541d7afba service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:33.853402 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fb772-288a-4a88-be25-489541d7afba service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:33.853709 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fb772-288a-4a88-be25-489541d7afba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:33.862901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fb772-288a-4a88-be25-489541d7afba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:33.864344 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fb772-288a-4a88-be25-489541d7afba service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:33.873798 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 774/1536] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:37:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:37:35.951717 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74fc66-6fec-4fa8-8cb2-eb1d173a7e83 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:35.952289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74fc66-6fec-4fa8-8cb2-eb1d173a7e83 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:35.952289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74fc66-6fec-4fa8-8cb2-eb1d173a7e83 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:35.953691 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:35.958969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bf84c-4bee-498d-96a8-bde81f90b7e0 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:35.959412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bf84c-4bee-498d-96a8-bde81f90b7e0 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:35.959531 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bf84c-4bee-498d-96a8-bde81f90b7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:35.977680 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-691bf84c-4bee-498d-96a8-bde81f90b7e0 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:35.979478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-691bf84c-4bee-498d-96a8-bde81f90b7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:35.987339 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:37:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:37:36.291919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74fc66-6fec-4fa8-8cb2-eb1d173a7e83 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:36.293932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-ed74fc66-6fec-4fa8-8cb2-eb1d173a7e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:36.306949 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74fc66-6fec-4fa8-8cb2-eb1d173a7e83 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:36.310450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0d32e-9334-4584-9feb-5738409933b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:36.311083 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0d32e-9334-4584-9feb-5738409933b2 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:36.311369 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0d32e-9334-4584-9feb-5738409933b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:36.311648 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0d32e-9334-4584-9feb-5738409933b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:36.320789 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0d32e-9334-4584-9feb-5738409933b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:36.322261 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0d32e-9334-4584-9feb-5738409933b2 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:36.331764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 764/1539] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:37:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:37:45.972815 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=11 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:46.327646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=89 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:07.838328 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5f948-2483-4119-8741-63f0cc72f5b7 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:07.839603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5f948-2483-4119-8741-63f0cc72f5b7 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:07.840134 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5f948-2483-4119-8741-63f0cc72f5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:07.865702 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-30c5f948-2483-4119-8741-63f0cc72f5b7 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:07.867282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-30c5f948-2483-4119-8741-63f0cc72f5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:07.881121 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 776/1540] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:07.890290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd592fd-b638-4733-b3af-7f6da158923c None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:07.891494 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd592fd-b638-4733-b3af-7f6da158923c None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:07.892000 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd592fd-b638-4733-b3af-7f6da158923c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:07.917904 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-ddd592fd-b638-4733-b3af-7f6da158923c None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:07.919516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-ddd592fd-b638-4733-b3af-7f6da158923c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:07.930470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 765/1541] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:07.935986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5f948-2483-4119-8741-63f0cc72f5b7 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:07.949799 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5f948-2483-4119-8741-63f0cc72f5b7 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:07.955600 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c7b7e-6bc2-4403-8cf7-001fc2142d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:07.956724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c7b7e-6bc2-4403-8cf7-001fc2142d63 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:07.957264 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c7b7e-6bc2-4403-8cf7-001fc2142d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:07.957770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c7b7e-6bc2-4403-8cf7-001fc2142d63 admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:07.958866 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c7b7e-6bc2-4403-8cf7-001fc2142d63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:07.959496 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c7b7e-6bc2-4403-8cf7-001fc2142d63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:07.961486 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c7b7e-6bc2-4403-8cf7-001fc2142d63 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:07.982541 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 777/1542] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:07.991353 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd592fd-b638-4733-b3af-7f6da158923c None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.002801 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd592fd-b638-4733-b3af-7f6da158923c None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.013217 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72e9be-9293-4e96-b11d-73982c7fd411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.014555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72e9be-9293-4e96-b11d-73982c7fd411 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.015111 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72e9be-9293-4e96-b11d-73982c7fd411 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.015628 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72e9be-9293-4e96-b11d-73982c7fd411 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.016666 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab72e9be-9293-4e96-b11d-73982c7fd411 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.019012 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab72e9be-9293-4e96-b11d-73982c7fd411 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.077641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 766/1543] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:07 2022] POST /identity/v3/projects => generated 373 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:08.081737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c7b7e-6bc2-4403-8cf7-001fc2142d63 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.095513 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c7b7e-6bc2-4403-8cf7-001fc2142d63 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.171831 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cccc14-c30f-4aa9-8e38-880bf58a29e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.172932 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cccc14-c30f-4aa9-8e38-880bf58a29e2 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.173444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cccc14-c30f-4aa9-8e38-880bf58a29e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.173949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cccc14-c30f-4aa9-8e38-880bf58a29e2 admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.174840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cccc14-c30f-4aa9-8e38-880bf58a29e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.177449 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cccc14-c30f-4aa9-8e38-880bf58a29e2 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.188485 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101bf9a-ccbf-482c-8064-0627865e8138 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.189036 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101bf9a-ccbf-482c-8064-0627865e8138 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.189513 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101bf9a-ccbf-482c-8064-0627865e8138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.228231 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-7101bf9a-ccbf-482c-8064-0627865e8138 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:08.230932 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 778/1544] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:08.232326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cccc14-c30f-4aa9-8e38-880bf58a29e2 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.234084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-7101bf9a-ccbf-482c-8064-0627865e8138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:08.257505 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cccc14-c30f-4aa9-8e38-880bf58a29e2 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.258184 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 767/1545] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:08.265508 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a681a3-fb49-4b32-a19e-5aa4af492a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.268621 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee646e0-db05-438d-9eeb-413a1552b3e1 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.269155 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee646e0-db05-438d-9eeb-413a1552b3e1 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.269603 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee646e0-db05-438d-9eeb-413a1552b3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.274458 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a681a3-fb49-4b32-a19e-5aa4af492a08 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.275626 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a681a3-fb49-4b32-a19e-5aa4af492a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.280269 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a681a3-fb49-4b32-a19e-5aa4af492a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.281383 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a681a3-fb49-4b32-a19e-5aa4af492a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.284461 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a681a3-fb49-4b32-a19e-5aa4af492a08 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.300532 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-fee646e0-db05-438d-9eeb-413a1552b3e1 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:08.303964 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-fee646e0-db05-438d-9eeb-413a1552b3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:08.309590 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 779/1546] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:08.315766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8556e6-c7cd-46d1-8439-d8eede8a1fff None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.315766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8556e6-c7cd-46d1-8439-d8eede8a1fff None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.315766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8556e6-c7cd-46d1-8439-d8eede8a1fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.325227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 768/1547] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:08.326852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee646e0-db05-438d-9eeb-413a1552b3e1 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.338994 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee646e0-db05-438d-9eeb-413a1552b3e1 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.350723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c438c75-52c1-4a14-9410-bcf7888fd870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.351721 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c438c75-52c1-4a14-9410-bcf7888fd870 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.352253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c438c75-52c1-4a14-9410-bcf7888fd870 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.352761 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c438c75-52c1-4a14-9410-bcf7888fd870 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.353165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-8e8556e6-c7cd-46d1-8439-d8eede8a1fff None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:08.354011 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c438c75-52c1-4a14-9410-bcf7888fd870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.354608 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-8e8556e6-c7cd-46d1-8439-d8eede8a1fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:08.356145 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c438c75-52c1-4a14-9410-bcf7888fd870 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.368494 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 769/1548] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:08.369722 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c438c75-52c1-4a14-9410-bcf7888fd870 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.381965 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c438c75-52c1-4a14-9410-bcf7888fd870 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.387079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 780/1549] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:08.387536 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd1d54-6727-4c59-b313-01f2dbbef943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.388261 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd1d54-6727-4c59-b313-01f2dbbef943 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.388396 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78842964-6036-45a4-92e2-8333dbea6ed7 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.388573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd1d54-6727-4c59-b313-01f2dbbef943 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.388685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78842964-6036-45a4-92e2-8333dbea6ed7 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.388842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd1d54-6727-4c59-b313-01f2dbbef943 admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.388960 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78842964-6036-45a4-92e2-8333dbea6ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.389603 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd1d54-6727-4c59-b313-01f2dbbef943 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:08.389910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd1d54-6727-4c59-b313-01f2dbbef943 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.390939 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: WARNING py.warnings [None req-3ecd1d54-6727-4c59-b313-01f2dbbef943 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:39:08.390939 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: warnings.warn(msg) Jul 21 20:39:08.390939 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: Jul 21 20:39:08.391832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd1d54-6727-4c59-b313-01f2dbbef943 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.406786 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 770/1550] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:08.408298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81dbde6-cce9-4350-930a-28e521be789c None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.408593 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81dbde6-cce9-4350-930a-28e521be789c None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.408871 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81dbde6-cce9-4350-930a-28e521be789c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.430150 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-d81dbde6-cce9-4350-930a-28e521be789c None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:08.431365 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d81dbde6-cce9-4350-930a-28e521be789c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:08.438696 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-78842964-6036-45a4-92e2-8333dbea6ed7 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:08.441672 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-78842964-6036-45a4-92e2-8333dbea6ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:08.447481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 771/1551] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:08.453257 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81dbde6-cce9-4350-930a-28e521be789c None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.461809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 781/1552] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:08.462873 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e90632-542e-4394-93ef-9c1e1b16cb07 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.463156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e90632-542e-4394-93ef-9c1e1b16cb07 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.463431 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e90632-542e-4394-93ef-9c1e1b16cb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.483276 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81dbde6-cce9-4350-930a-28e521be789c None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.490024 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3fb42-4ca4-4dcc-b381-15c626fe5079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.491018 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3fb42-4ca4-4dcc-b381-15c626fe5079 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.491304 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3fb42-4ca4-4dcc-b381-15c626fe5079 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.491583 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3fb42-4ca4-4dcc-b381-15c626fe5079 admin admin] PATH_INFO: `/v3/projects/2b66f19119de450b9e7fee99b7918e30/users/226f6f56564b42e8be142a727efc37e6/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.497582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-04e90632-542e-4394-93ef-9c1e1b16cb07 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:08.498797 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-04e90632-542e-4394-93ef-9c1e1b16cb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:08.506987 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 782/1553] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:08.507807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e90632-542e-4394-93ef-9c1e1b16cb07 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.508244 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3fb42-4ca4-4dcc-b381-15c626fe5079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b66f19119de450b9e7fee99b7918e30, user_id=226f6f56564b42e8be142a727efc37e6, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.511639 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3fb42-4ca4-4dcc-b381-15c626fe5079 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.518814 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e90632-542e-4394-93ef-9c1e1b16cb07 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.525208 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 772/1554] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:08 2022] PUT /identity/v3/projects/2b66f19119de450b9e7fee99b7918e30/users/226f6f56564b42e8be142a727efc37e6/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:08.526523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acc082-1e8c-4b12-b89d-b8c1c767f56e None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.526819 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acc082-1e8c-4b12-b89d-b8c1c767f56e None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.527097 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acc082-1e8c-4b12-b89d-b8c1c767f56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.544853 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-22acc082-1e8c-4b12-b89d-b8c1c767f56e None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:08.546106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-22acc082-1e8c-4b12-b89d-b8c1c767f56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:08.583929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d9899-0fe7-433c-b626-62ec1442a586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.584589 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d9899-0fe7-433c-b626-62ec1442a586 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.584871 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d9899-0fe7-433c-b626-62ec1442a586 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.585174 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d9899-0fe7-433c-b626-62ec1442a586 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.585867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d9899-0fe7-433c-b626-62ec1442a586 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.586891 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: WARNING py.warnings [None req-6b8d9899-0fe7-433c-b626-62ec1442a586 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:39:08.586891 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: warnings.warn(msg) Jul 21 20:39:08.586891 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: Jul 21 20:39:08.588080 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d9899-0fe7-433c-b626-62ec1442a586 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.608413 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 773/1555] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:08.609616 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4d6a1-2b61-4a57-9510-3d1b6f7762c3 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.609913 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4d6a1-2b61-4a57-9510-3d1b6f7762c3 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.610193 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4d6a1-2b61-4a57-9510-3d1b6f7762c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.617531 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 783/1556] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:08.618892 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c44450-89c0-4be7-9a25-e7314793a0fc None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.619188 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c44450-89c0-4be7-9a25-e7314793a0fc None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.619473 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c44450-89c0-4be7-9a25-e7314793a0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.632722 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-f3a4d6a1-2b61-4a57-9510-3d1b6f7762c3 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:08.633972 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-f3a4d6a1-2b61-4a57-9510-3d1b6f7762c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:08.637991 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-00c44450-89c0-4be7-9a25-e7314793a0fc None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:08.639165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-00c44450-89c0-4be7-9a25-e7314793a0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:08.659344 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:08.687495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:08.700597 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 774/1557] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:08.701990 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe77521-41cd-47f6-b613-e8bb5d47e76c None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.709340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe77521-41cd-47f6-b613-e8bb5d47e76c None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.709501 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 784/1558] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:08.709633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe77521-41cd-47f6-b613-e8bb5d47e76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.709748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a84cdb-50a1-4195-87f8-ca1d45551c0f None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.709748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a84cdb-50a1-4195-87f8-ca1d45551c0f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.709748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a84cdb-50a1-4195-87f8-ca1d45551c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.725506 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-ebe77521-41cd-47f6-b613-e8bb5d47e76c None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:08.725902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-a6a84cdb-50a1-4195-87f8-ca1d45551c0f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:08.726716 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-ebe77521-41cd-47f6-b613-e8bb5d47e76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:08.727045 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a6a84cdb-50a1-4195-87f8-ca1d45551c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:08.734628 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 775/1559] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:08.735094 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 785/1560] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:08.735416 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe77521-41cd-47f6-b613-e8bb5d47e76c None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.736051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b34070-d502-4e6d-bd0a-b751409d52c0 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.736334 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b34070-d502-4e6d-bd0a-b751409d52c0 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.736599 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b34070-d502-4e6d-bd0a-b751409d52c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.745057 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe77521-41cd-47f6-b613-e8bb5d47e76c None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.750715 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584feae-098f-431e-9f97-c8a6e8df56b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.751382 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584feae-098f-431e-9f97-c8a6e8df56b5 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.751667 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584feae-098f-431e-9f97-c8a6e8df56b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.751945 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584feae-098f-431e-9f97-c8a6e8df56b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.752723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584feae-098f-431e-9f97-c8a6e8df56b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:08.753035 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584feae-098f-431e-9f97-c8a6e8df56b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.754667 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584feae-098f-431e-9f97-c8a6e8df56b5 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.766932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 776/1561] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:08.774639 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584feae-098f-431e-9f97-c8a6e8df56b5 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.774778 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-a2b34070-d502-4e6d-bd0a-b751409d52c0 None None] MFA Rules not processed for user `226f6f56564b42e8be142a727efc37e6`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:08.774778 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a2b34070-d502-4e6d-bd0a-b751409d52c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:08.779985 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584feae-098f-431e-9f97-c8a6e8df56b5 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.784609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ce247-49ef-4f7e-ba5f-50a77eb7ac94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.785285 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ce247-49ef-4f7e-ba5f-50a77eb7ac94 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.785573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ce247-49ef-4f7e-ba5f-50a77eb7ac94 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.785849 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ce247-49ef-4f7e-ba5f-50a77eb7ac94 admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.786934 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ce247-49ef-4f7e-ba5f-50a77eb7ac94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:08.786934 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ce247-49ef-4f7e-ba5f-50a77eb7ac94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.788463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ce247-49ef-4f7e-ba5f-50a77eb7ac94 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.801607 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 777/1562] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:08.802537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ce247-49ef-4f7e-ba5f-50a77eb7ac94 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.815382 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ce247-49ef-4f7e-ba5f-50a77eb7ac94 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.820003 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b78dd9-bef7-42cb-af74-598db9e61017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.820663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b78dd9-bef7-42cb-af74-598db9e61017 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.820947 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b78dd9-bef7-42cb-af74-598db9e61017 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.821257 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b78dd9-bef7-42cb-af74-598db9e61017 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.821989 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b78dd9-bef7-42cb-af74-598db9e61017 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.823009 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: WARNING py.warnings [None req-a5b78dd9-bef7-42cb-af74-598db9e61017 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:39:08.823009 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: warnings.warn(msg) Jul 21 20:39:08.823009 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: Jul 21 20:39:08.824216 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b78dd9-bef7-42cb-af74-598db9e61017 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.838125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 786/1563] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:08.839430 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403bb43-0f79-4bde-8cbb-c815a46d56b2 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.840052 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403bb43-0f79-4bde-8cbb-c815a46d56b2 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.840052 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403bb43-0f79-4bde-8cbb-c815a46d56b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.850403 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 778/1564] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/users => generated 290 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:08.850403 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d8b2e-5511-4839-ad73-e16524861877 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.850403 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d8b2e-5511-4839-ad73-e16524861877 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.850403 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d8b2e-5511-4839-ad73-e16524861877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.862844 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-a403bb43-0f79-4bde-8cbb-c815a46d56b2 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:08.863957 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a403bb43-0f79-4bde-8cbb-c815a46d56b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:08.867773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-281d8b2e-5511-4839-ad73-e16524861877 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:08.868945 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-281d8b2e-5511-4839-ad73-e16524861877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:08.872600 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 787/1565] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:08.873471 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403bb43-0f79-4bde-8cbb-c815a46d56b2 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.881996 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 779/1566] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:08.883465 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e2e0a-99db-466a-b2a4-e313dc63065d None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.883751 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e2e0a-99db-466a-b2a4-e313dc63065d None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.884028 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e2e0a-99db-466a-b2a4-e313dc63065d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.886013 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403bb43-0f79-4bde-8cbb-c815a46d56b2 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.890869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caac95-95a5-4c81-a219-ad33e8cf8fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.891557 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caac95-95a5-4c81-a219-ad33e8cf8fbb admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.891845 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caac95-95a5-4c81-a219-ad33e8cf8fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.892119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caac95-95a5-4c81-a219-ad33e8cf8fbb admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.892863 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94caac95-95a5-4c81-a219-ad33e8cf8fbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:08.893227 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94caac95-95a5-4c81-a219-ad33e8cf8fbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.895256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94caac95-95a5-4c81-a219-ad33e8cf8fbb admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.903115 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-530e2e0a-99db-466a-b2a4-e313dc63065d None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:08.904331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-530e2e0a-99db-466a-b2a4-e313dc63065d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:08.908581 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 788/1567] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:08.909508 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caac95-95a5-4c81-a219-ad33e8cf8fbb admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.916362 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 780/1568] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:08.916362 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e2e0a-99db-466a-b2a4-e313dc63065d None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.920331 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caac95-95a5-4c81-a219-ad33e8cf8fbb admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.924947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22886e94-3435-4ca9-b2b8-00235447fbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.925650 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22886e94-3435-4ca9-b2b8-00235447fbf8 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.925937 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22886e94-3435-4ca9-b2b8-00235447fbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.926218 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22886e94-3435-4ca9-b2b8-00235447fbf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.926470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e2e0a-99db-466a-b2a4-e313dc63065d None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.926909 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22886e94-3435-4ca9-b2b8-00235447fbf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.928826 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22886e94-3435-4ca9-b2b8-00235447fbf8 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.930993 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cac0c0-3cfa-4e14-8bf9-b41b674bfca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.931621 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cac0c0-3cfa-4e14-8bf9-b41b674bfca4 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.931939 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cac0c0-3cfa-4e14-8bf9-b41b674bfca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.932215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cac0c0-3cfa-4e14-8bf9-b41b674bfca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.933699 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cac0c0-3cfa-4e14-8bf9-b41b674bfca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.935380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cac0c0-3cfa-4e14-8bf9-b41b674bfca4 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.956084 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 789/1569] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:08.956895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22886e94-3435-4ca9-b2b8-00235447fbf8 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.961906 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 781/1570] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:08.963253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cac0c0-3cfa-4e14-8bf9-b41b674bfca4 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.021803 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22886e94-3435-4ca9-b2b8-00235447fbf8 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.026360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2f1e1-9636-402b-b9d8-4f085e95227e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.027014 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2f1e1-9636-402b-b9d8-4f085e95227e admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.027294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2f1e1-9636-402b-b9d8-4f085e95227e admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.027564 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2f1e1-9636-402b-b9d8-4f085e95227e admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.028249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2f1e1-9636-402b-b9d8-4f085e95227e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.028372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cac0c0-3cfa-4e14-8bf9-b41b674bfca4 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.030184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2f1e1-9636-402b-b9d8-4f085e95227e admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.032891 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dea5e0-cc9b-4c29-a9e0-99453138bd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.033654 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dea5e0-cc9b-4c29-a9e0-99453138bd6f admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.033937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dea5e0-cc9b-4c29-a9e0-99453138bd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.034217 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dea5e0-cc9b-4c29-a9e0-99453138bd6f admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.034953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dea5e0-cc9b-4c29-a9e0-99453138bd6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.036546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dea5e0-cc9b-4c29-a9e0-99453138bd6f admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.040179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:09.048033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 782/1571] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:09.049027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dea5e0-cc9b-4c29-a9e0-99453138bd6f admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.059644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dea5e0-cc9b-4c29-a9e0-99453138bd6f admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.063272 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 790/1572] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:08 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:09.063272 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2f1e1-9636-402b-b9d8-4f085e95227e admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.121322 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b082a-ccdb-4a5f-a18f-9e70d80dc34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.121910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b082a-ccdb-4a5f-a18f-9e70d80dc34c admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.122195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b082a-ccdb-4a5f-a18f-9e70d80dc34c admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.122478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b082a-ccdb-4a5f-a18f-9e70d80dc34c admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.123235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b082a-ccdb-4a5f-a18f-9e70d80dc34c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:09.123542 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b082a-ccdb-4a5f-a18f-9e70d80dc34c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.125181 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b082a-ccdb-4a5f-a18f-9e70d80dc34c admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.128806 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2f1e1-9636-402b-b9d8-4f085e95227e admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.133488 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29f836-66dc-4720-942d-46ddcc7bbde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.134173 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29f836-66dc-4720-942d-46ddcc7bbde2 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.134454 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29f836-66dc-4720-942d-46ddcc7bbde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.134856 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29f836-66dc-4720-942d-46ddcc7bbde2 admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.134973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:09.135887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29f836-66dc-4720-942d-46ddcc7bbde2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:09.136062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29f836-66dc-4720-942d-46ddcc7bbde2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.137566 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 783/1573] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:09.138221 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b082a-ccdb-4a5f-a18f-9e70d80dc34c admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.138335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29f836-66dc-4720-942d-46ddcc7bbde2 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.149213 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b082a-ccdb-4a5f-a18f-9e70d80dc34c admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.155679 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790a66f-e3a9-4d1a-8f4b-d4fa84b52554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.156256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 791/1574] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:09.156548 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790a66f-e3a9-4d1a-8f4b-d4fa84b52554 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.156651 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790a66f-e3a9-4d1a-8f4b-d4fa84b52554 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.156892 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790a66f-e3a9-4d1a-8f4b-d4fa84b52554 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.157610 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790a66f-e3a9-4d1a-8f4b-d4fa84b52554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.158153 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29f836-66dc-4720-942d-46ddcc7bbde2 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.159769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790a66f-e3a9-4d1a-8f4b-d4fa84b52554 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.169507 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29f836-66dc-4720-942d-46ddcc7bbde2 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.174934 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ce5ae-bfd4-44ed-9256-16438b8de517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.175584 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ce5ae-bfd4-44ed-9256-16438b8de517 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.175865 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ce5ae-bfd4-44ed-9256-16438b8de517 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.176144 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ce5ae-bfd4-44ed-9256-16438b8de517 admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.176960 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ce5ae-bfd4-44ed-9256-16438b8de517 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:09.177317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ce5ae-bfd4-44ed-9256-16438b8de517 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.179310 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ce5ae-bfd4-44ed-9256-16438b8de517 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.187196 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 784/1575] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:09 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:09.187996 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790a66f-e3a9-4d1a-8f4b-d4fa84b52554 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.201060 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 792/1576] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:09.201060 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ce5ae-bfd4-44ed-9256-16438b8de517 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.204699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ce5ae-bfd4-44ed-9256-16438b8de517 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.257677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790a66f-e3a9-4d1a-8f4b-d4fa84b52554 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.261947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eea9eb-0cdb-4fd7-a8f9-a2ebc6978893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.262353 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe879d-3ea1-4ea2-ab37-4e11f39c2330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.262612 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eea9eb-0cdb-4fd7-a8f9-a2ebc6978893 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.262891 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eea9eb-0cdb-4fd7-a8f9-a2ebc6978893 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.263011 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe879d-3ea1-4ea2-ab37-4e11f39c2330 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.263159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eea9eb-0cdb-4fd7-a8f9-a2ebc6978893 admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.263290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe879d-3ea1-4ea2-ab37-4e11f39c2330 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.263551 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe879d-3ea1-4ea2-ab37-4e11f39c2330 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.263846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eea9eb-0cdb-4fd7-a8f9-a2ebc6978893 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.264274 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe879d-3ea1-4ea2-ab37-4e11f39c2330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.265793 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eea9eb-0cdb-4fd7-a8f9-a2ebc6978893 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.266215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe879d-3ea1-4ea2-ab37-4e11f39c2330 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.287338 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 793/1577] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:09 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:09.288172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eea9eb-0cdb-4fd7-a8f9-a2ebc6978893 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.292193 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 785/1578] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:09 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:09.292193 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe879d-3ea1-4ea2-ab37-4e11f39c2330 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.300249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe879d-3ea1-4ea2-ab37-4e11f39c2330 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.303401 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eea9eb-0cdb-4fd7-a8f9-a2ebc6978893 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.304965 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6c69b-bc43-43a8-ba9c-1870a977e346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.305626 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6c69b-bc43-43a8-ba9c-1870a977e346 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.305909 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6c69b-bc43-43a8-ba9c-1870a977e346 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.306183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6c69b-bc43-43a8-ba9c-1870a977e346 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.306903 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6c69b-bc43-43a8-ba9c-1870a977e346 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.308663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6004a-62f9-486a-a7be-f627c6da3304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.308868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6c69b-bc43-43a8-ba9c-1870a977e346 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.309347 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6004a-62f9-486a-a7be-f627c6da3304 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.309631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6004a-62f9-486a-a7be-f627c6da3304 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.309907 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6004a-62f9-486a-a7be-f627c6da3304 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.310628 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6004a-62f9-486a-a7be-f627c6da3304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.312214 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6004a-62f9-486a-a7be-f627c6da3304 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.324288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 794/1579] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:09.325109 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6004a-62f9-486a-a7be-f627c6da3304 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.332575 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 786/1580] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:09 2022] POST /identity/v3/users => generated 293 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:09.333408 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6c69b-bc43-43a8-ba9c-1870a977e346 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.335095 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6004a-62f9-486a-a7be-f627c6da3304 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.340155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128a528-fdd6-44c0-a4d4-d0d8d24a3a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.340803 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128a528-fdd6-44c0-a4d4-d0d8d24a3a69 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.341108 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128a528-fdd6-44c0-a4d4-d0d8d24a3a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.341388 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128a528-fdd6-44c0-a4d4-d0d8d24a3a69 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.342071 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128a528-fdd6-44c0-a4d4-d0d8d24a3a69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.343864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6c69b-bc43-43a8-ba9c-1870a977e346 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.343987 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128a528-fdd6-44c0-a4d4-d0d8d24a3a69 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.348906 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d127e98-0720-4e86-9d51-135375a8b35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.349587 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d127e98-0720-4e86-9d51-135375a8b35c admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.349889 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d127e98-0720-4e86-9d51-135375a8b35c admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.350155 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d127e98-0720-4e86-9d51-135375a8b35c admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.350897 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d127e98-0720-4e86-9d51-135375a8b35c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.353039 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d127e98-0720-4e86-9d51-135375a8b35c admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.370921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 795/1581] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:09 2022] POST /identity/v3/projects => generated 357 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:09.374422 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128a528-fdd6-44c0-a4d4-d0d8d24a3a69 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.382031 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 787/1582] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:09 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:09.382912 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d127e98-0720-4e86-9d51-135375a8b35c admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.403749 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:09.440257 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128a528-fdd6-44c0-a4d4-d0d8d24a3a69 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.445241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00391b9-14a4-45b3-b251-7a7f75daabda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.445912 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00391b9-14a4-45b3-b251-7a7f75daabda admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.446195 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00391b9-14a4-45b3-b251-7a7f75daabda admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.446468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00391b9-14a4-45b3-b251-7a7f75daabda admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.447156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00391b9-14a4-45b3-b251-7a7f75daabda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.449289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00391b9-14a4-45b3-b251-7a7f75daabda admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.450766 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d127e98-0720-4e86-9d51-135375a8b35c admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.455871 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e2c77-550f-431e-9386-bc930b25c8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.456635 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e2c77-550f-431e-9386-bc930b25c8bb admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.456857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e2c77-550f-431e-9386-bc930b25c8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.457177 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e2c77-550f-431e-9386-bc930b25c8bb admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.457875 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e2c77-550f-431e-9386-bc930b25c8bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.459857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e2c77-550f-431e-9386-bc930b25c8bb admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.479212 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 796/1583] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:09 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:09.479212 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00391b9-14a4-45b3-b251-7a7f75daabda admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.488817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00391b9-14a4-45b3-b251-7a7f75daabda admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.495085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 788/1584] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:09 2022] POST /identity/v3/projects => generated 367 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:09.495085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e2c77-550f-431e-9386-bc930b25c8bb admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.495387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3441b-faca-4981-a4eb-a23dd844b39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.495387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3441b-faca-4981-a4eb-a23dd844b39c admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.495387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3441b-faca-4981-a4eb-a23dd844b39c admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.495387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3441b-faca-4981-a4eb-a23dd844b39c admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.495769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d3441b-faca-4981-a4eb-a23dd844b39c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.497353 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d3441b-faca-4981-a4eb-a23dd844b39c admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.507611 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:09.507813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 797/1585] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:09.508629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3441b-faca-4981-a4eb-a23dd844b39c admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.519731 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3441b-faca-4981-a4eb-a23dd844b39c admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.561177 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e2c77-550f-431e-9386-bc930b25c8bb admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.566446 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582e9ca-67ae-4c61-9648-79e057935021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.567117 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582e9ca-67ae-4c61-9648-79e057935021 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.567395 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582e9ca-67ae-4c61-9648-79e057935021 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.567666 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582e9ca-67ae-4c61-9648-79e057935021 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.568397 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582e9ca-67ae-4c61-9648-79e057935021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.570021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582e9ca-67ae-4c61-9648-79e057935021 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.577359 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6855bdc-0462-48aa-851c-d4eae22431dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.578263 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6855bdc-0462-48aa-851c-d4eae22431dd admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.578528 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6855bdc-0462-48aa-851c-d4eae22431dd admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.579309 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6855bdc-0462-48aa-851c-d4eae22431dd admin admin] PATH_INFO: `/v3/projects/620b43f77db64f51a974bce26c1417c3/users/66d711e34087402abd12dae6113f6053/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.581048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 789/1586] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:09.581870 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582e9ca-67ae-4c61-9648-79e057935021 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.591363 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582e9ca-67ae-4c61-9648-79e057935021 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.596332 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3a6cd-c2de-4e3d-957a-8f4ae6c20c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.596970 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3a6cd-c2de-4e3d-957a-8f4ae6c20c94 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.597239 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6855bdc-0462-48aa-851c-d4eae22431dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=620b43f77db64f51a974bce26c1417c3, user_id=66d711e34087402abd12dae6113f6053, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.597361 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3a6cd-c2de-4e3d-957a-8f4ae6c20c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.597548 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3a6cd-c2de-4e3d-957a-8f4ae6c20c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.598418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: WARNING py.warnings [None req-f6855bdc-0462-48aa-851c-d4eae22431dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:39:09.598418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: warnings.warn(msg) Jul 21 20:39:09.598418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: Jul 21 20:39:09.598817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3a6cd-c2de-4e3d-957a-8f4ae6c20c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.599859 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3a6cd-c2de-4e3d-957a-8f4ae6c20c94 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.600254 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6855bdc-0462-48aa-851c-d4eae22431dd admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.610704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 790/1587] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:09.613997 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3a6cd-c2de-4e3d-957a-8f4ae6c20c94 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.614421 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 798/1588] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:09 2022] PUT /identity/v3/projects/620b43f77db64f51a974bce26c1417c3/users/66d711e34087402abd12dae6113f6053/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:09.615283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6855bdc-0462-48aa-851c-d4eae22431dd admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.623976 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3a6cd-c2de-4e3d-957a-8f4ae6c20c94 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.688597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6855bdc-0462-48aa-851c-d4eae22431dd admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.689865 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba945b-86ce-4250-8a42-4711312876e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.690577 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba945b-86ce-4250-8a42-4711312876e7 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.690855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba945b-86ce-4250-8a42-4711312876e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.691132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba945b-86ce-4250-8a42-4711312876e7 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.691830 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba945b-86ce-4250-8a42-4711312876e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.693523 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5033b4-ae10-4fee-a782-20304b369771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.693804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba945b-86ce-4250-8a42-4711312876e7 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.694186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5033b4-ae10-4fee-a782-20304b369771 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.694494 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5033b4-ae10-4fee-a782-20304b369771 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.694770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5033b4-ae10-4fee-a782-20304b369771 admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.695451 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5033b4-ae10-4fee-a782-20304b369771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.697395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5033b4-ae10-4fee-a782-20304b369771 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.716555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 791/1589] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:09 2022] POST /identity/v3/users => generated 279 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:09.717420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba945b-86ce-4250-8a42-4711312876e7 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.720538 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 799/1590] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:09 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:09.721505 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5033b4-ae10-4fee-a782-20304b369771 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.728566 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba945b-86ce-4250-8a42-4711312876e7 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.731432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5033b4-ae10-4fee-a782-20304b369771 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.733469 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deea063-72ef-4611-926a-f4cd7460feaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.734188 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deea063-72ef-4611-926a-f4cd7460feaf admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.734476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deea063-72ef-4611-926a-f4cd7460feaf admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.734757 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deea063-72ef-4611-926a-f4cd7460feaf admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.735505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deea063-72ef-4611-926a-f4cd7460feaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.737118 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deea063-72ef-4611-926a-f4cd7460feaf admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.737250 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbb3f8-d2b7-4f27-853b-3f2034a384ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.737889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbb3f8-d2b7-4f27-853b-3f2034a384ff admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.738170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbb3f8-d2b7-4f27-853b-3f2034a384ff admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.738445 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbb3f8-d2b7-4f27-853b-3f2034a384ff admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.739128 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdbb3f8-d2b7-4f27-853b-3f2034a384ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.741030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdbb3f8-d2b7-4f27-853b-3f2034a384ff admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.748609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 792/1591] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:09.749368 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deea063-72ef-4611-926a-f4cd7460feaf admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.760302 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deea063-72ef-4611-926a-f4cd7460feaf admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.762800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 800/1592] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:09 2022] POST /identity/v3/users => generated 284 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:09.763737 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbb3f8-d2b7-4f27-853b-3f2034a384ff admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.768565 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a7fd6-c7a3-480c-9b7a-948c4a4a1e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.769259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a7fd6-c7a3-480c-9b7a-948c4a4a1e6b admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.769560 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a7fd6-c7a3-480c-9b7a-948c4a4a1e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.769828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a7fd6-c7a3-480c-9b7a-948c4a4a1e6b admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.770607 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a7fd6-c7a3-480c-9b7a-948c4a4a1e6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.771295 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:09.772213 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a7fd6-c7a3-480c-9b7a-948c4a4a1e6b admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.777923 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbb3f8-d2b7-4f27-853b-3f2034a384ff admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.783033 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd2ba1-3959-4eee-ba50-d1107953c4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.786740 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 793/1593] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:09.786907 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd2ba1-3959-4eee-ba50-d1107953c4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.786907 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd2ba1-3959-4eee-ba50-d1107953c4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.786907 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd2ba1-3959-4eee-ba50-d1107953c4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.787201 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a7fd6-c7a3-480c-9b7a-948c4a4a1e6b admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.787281 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd2ba1-3959-4eee-ba50-d1107953c4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.787281 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd2ba1-3959-4eee-ba50-d1107953c4e7 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.797523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a7fd6-c7a3-480c-9b7a-948c4a4a1e6b admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.797696 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 801/1594] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:09.798825 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede7839-a980-4cd9-a237-da3cfd9b54e8 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.799113 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede7839-a980-4cd9-a237-da3cfd9b54e8 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.799391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede7839-a980-4cd9-a237-da3cfd9b54e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.802613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b463610-b7d0-4cde-a16a-8c4cf0278e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.803510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b463610-b7d0-4cde-a16a-8c4cf0278e79 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.803795 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b463610-b7d0-4cde-a16a-8c4cf0278e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.804070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b463610-b7d0-4cde-a16a-8c4cf0278e79 admin admin] PATH_INFO: `/v3/projects/f03c2c0a06b64001b77bcef997e15a6e/users/ee2c89a61713457c9e3febf7ad85a1a9/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.819037 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b463610-b7d0-4cde-a16a-8c4cf0278e79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f03c2c0a06b64001b77bcef997e15a6e, user_id=ee2c89a61713457c9e3febf7ad85a1a9, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.821742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b463610-b7d0-4cde-a16a-8c4cf0278e79 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.825487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-bede7839-a980-4cd9-a237-da3cfd9b54e8 None None] MFA Rules not processed for user `66d711e34087402abd12dae6113f6053`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:09.826733 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-bede7839-a980-4cd9-a237-da3cfd9b54e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:09.835703 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 794/1595] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:09 2022] PUT /identity/v3/projects/f03c2c0a06b64001b77bcef997e15a6e/users/ee2c89a61713457c9e3febf7ad85a1a9/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:09.836800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b463610-b7d0-4cde-a16a-8c4cf0278e79 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.883114 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:09.899819 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 802/1596] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:09.900637 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede7839-a980-4cd9-a237-da3cfd9b54e8 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.901336 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b463610-b7d0-4cde-a16a-8c4cf0278e79 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.906597 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82626f-05f4-41e6-af17-541472686b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.907268 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82626f-05f4-41e6-af17-541472686b08 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.907555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82626f-05f4-41e6-af17-541472686b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.907859 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82626f-05f4-41e6-af17-541472686b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.908749 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82626f-05f4-41e6-af17-541472686b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.910412 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82626f-05f4-41e6-af17-541472686b08 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.911633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede7839-a980-4cd9-a237-da3cfd9b54e8 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.916133 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc45501-f809-4c54-975d-23ea009baa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.916775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc45501-f809-4c54-975d-23ea009baa89 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.917054 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc45501-f809-4c54-975d-23ea009baa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.918012 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc45501-f809-4c54-975d-23ea009baa89 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.918739 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc45501-f809-4c54-975d-23ea009baa89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.920301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc45501-f809-4c54-975d-23ea009baa89 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.924285 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 795/1597] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:09.925208 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82626f-05f4-41e6-af17-541472686b08 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.933868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 803/1598] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:09.933868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc45501-f809-4c54-975d-23ea009baa89 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.938180 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82626f-05f4-41e6-af17-541472686b08 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.941488 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc45501-f809-4c54-975d-23ea009baa89 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.942832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78273a48-c00e-4c11-98fa-dbbfa543ffbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.943485 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78273a48-c00e-4c11-98fa-dbbfa543ffbc admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.943783 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78273a48-c00e-4c11-98fa-dbbfa543ffbc admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.944045 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78273a48-c00e-4c11-98fa-dbbfa543ffbc admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.944778 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78273a48-c00e-4c11-98fa-dbbfa543ffbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.945939 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d74d63-f834-4dd8-b026-9976001b230d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.946411 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78273a48-c00e-4c11-98fa-dbbfa543ffbc admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.946640 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d74d63-f834-4dd8-b026-9976001b230d admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.946859 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d74d63-f834-4dd8-b026-9976001b230d admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.947131 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d74d63-f834-4dd8-b026-9976001b230d admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.947837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d74d63-f834-4dd8-b026-9976001b230d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.949529 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d74d63-f834-4dd8-b026-9976001b230d admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.959870 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 796/1599] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:09.959870 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78273a48-c00e-4c11-98fa-dbbfa543ffbc admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.960136 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 804/1600] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:09.960485 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d74d63-f834-4dd8-b026-9976001b230d admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.973549 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78273a48-c00e-4c11-98fa-dbbfa543ffbc admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.974583 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d74d63-f834-4dd8-b026-9976001b230d admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.978444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169da4d-5b30-4a08-a075-a6d0163fca6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.979466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169da4d-5b30-4a08-a075-a6d0163fca6c admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.980384 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169da4d-5b30-4a08-a075-a6d0163fca6c admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.980509 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93458376-92a6-47b0-b837-63892de97ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.980628 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169da4d-5b30-4a08-a075-a6d0163fca6c admin admin] PATH_INFO: `/v3/projects/f88b4bb8bbb54621ad28c6a75a3d2837/users/e971b95ceacc43d2ae82da35ec9f7fe5/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.981393 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93458376-92a6-47b0-b837-63892de97ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.981647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93458376-92a6-47b0-b837-63892de97ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.981919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93458376-92a6-47b0-b837-63892de97ebe admin admin] PATH_INFO: `/v3/projects/766a80d45e5144ac8317011c55113d46/users/654b47930e6f4a8ca378d6876cc8ae6c/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.996266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169da4d-5b30-4a08-a075-a6d0163fca6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f88b4bb8bbb54621ad28c6a75a3d2837, user_id=e971b95ceacc43d2ae82da35ec9f7fe5, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.998530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93458376-92a6-47b0-b837-63892de97ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=766a80d45e5144ac8317011c55113d46, user_id=654b47930e6f4a8ca378d6876cc8ae6c, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.999122 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169da4d-5b30-4a08-a075-a6d0163fca6c admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.001361 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93458376-92a6-47b0-b837-63892de97ebe admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.014819 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 797/1601] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:09 2022] PUT /identity/v3/projects/f88b4bb8bbb54621ad28c6a75a3d2837/users/e971b95ceacc43d2ae82da35ec9f7fe5/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:10.019338 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 805/1602] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:09 2022] PUT /identity/v3/projects/766a80d45e5144ac8317011c55113d46/users/654b47930e6f4a8ca378d6876cc8ae6c/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:10.019491 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169da4d-5b30-4a08-a075-a6d0163fca6c admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.019595 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93458376-92a6-47b0-b837-63892de97ebe admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.081817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93458376-92a6-47b0-b837-63892de97ebe admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.083767 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169da4d-5b30-4a08-a075-a6d0163fca6c admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.086516 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967842f-8c7f-4dc4-b561-b79a6b0e3468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.087176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967842f-8c7f-4dc4-b561-b79a6b0e3468 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.087458 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967842f-8c7f-4dc4-b561-b79a6b0e3468 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.087741 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967842f-8c7f-4dc4-b561-b79a6b0e3468 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.088459 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2967842f-8c7f-4dc4-b561-b79a6b0e3468 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.089517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afc797-fc65-47e1-81ab-0325538dafd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.090192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afc797-fc65-47e1-81ab-0325538dafd2 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.090477 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afc797-fc65-47e1-81ab-0325538dafd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.090752 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afc797-fc65-47e1-81ab-0325538dafd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.090957 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2967842f-8c7f-4dc4-b561-b79a6b0e3468 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.091518 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afc797-fc65-47e1-81ab-0325538dafd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.093131 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afc797-fc65-47e1-81ab-0325538dafd2 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.103597 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 798/1603] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:10.104421 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afc797-fc65-47e1-81ab-0325538dafd2 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.116831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afc797-fc65-47e1-81ab-0325538dafd2 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.118049 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 806/1604] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:10 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:10.122029 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967842f-8c7f-4dc4-b561-b79a6b0e3468 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.143812 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:10.177196 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade3fbe-78f4-497e-b270-9c68d3997849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.178043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade3fbe-78f4-497e-b270-9c68d3997849 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.178430 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade3fbe-78f4-497e-b270-9c68d3997849 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.178782 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade3fbe-78f4-497e-b270-9c68d3997849 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.179528 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade3fbe-78f4-497e-b270-9c68d3997849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.181156 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade3fbe-78f4-497e-b270-9c68d3997849 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.188668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967842f-8c7f-4dc4-b561-b79a6b0e3468 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.192757 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 799/1605] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:10.193368 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade3fbe-78f4-497e-b270-9c68d3997849 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.196418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464af7bd-a89f-4241-88af-3040af8ffddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.197107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464af7bd-a89f-4241-88af-3040af8ffddf admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.197390 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464af7bd-a89f-4241-88af-3040af8ffddf admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.197661 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464af7bd-a89f-4241-88af-3040af8ffddf admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.198464 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464af7bd-a89f-4241-88af-3040af8ffddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.200583 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464af7bd-a89f-4241-88af-3040af8ffddf admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.204075 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade3fbe-78f4-497e-b270-9c68d3997849 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.208678 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b1a73-bafd-4b8d-8375-fff291c0b5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.209609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b1a73-bafd-4b8d-8375-fff291c0b5f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.209891 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b1a73-bafd-4b8d-8375-fff291c0b5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.210168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b1a73-bafd-4b8d-8375-fff291c0b5f9 admin admin] PATH_INFO: `/v3/projects/d821e52798ef48c2b8a8cae87cfddaf5/users/5fc245eb98024dd382f2d2e72ee439de/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.214294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 807/1606] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:10.214294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464af7bd-a89f-4241-88af-3040af8ffddf admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.223046 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464af7bd-a89f-4241-88af-3040af8ffddf admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.227841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb140deb-9477-4262-aa57-a52346e159ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.228004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b1a73-bafd-4b8d-8375-fff291c0b5f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d821e52798ef48c2b8a8cae87cfddaf5, user_id=5fc245eb98024dd382f2d2e72ee439de, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.228520 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb140deb-9477-4262-aa57-a52346e159ba admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.228800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb140deb-9477-4262-aa57-a52346e159ba admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.229111 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb140deb-9477-4262-aa57-a52346e159ba admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.229853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb140deb-9477-4262-aa57-a52346e159ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.230679 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b1a73-bafd-4b8d-8375-fff291c0b5f9 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.231437 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb140deb-9477-4262-aa57-a52346e159ba admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.242319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 808/1607] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:10.243845 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e7fda-958d-478d-a19e-c51f2710ab43 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.244188 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e7fda-958d-478d-a19e-c51f2710ab43 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.244560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e7fda-958d-478d-a19e-c51f2710ab43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.249804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 800/1608] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:10 2022] PUT /identity/v3/projects/d821e52798ef48c2b8a8cae87cfddaf5/users/5fc245eb98024dd382f2d2e72ee439de/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:10.249804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014121e4-4ccc-4b00-9c3a-1c28062e21a5 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.249804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014121e4-4ccc-4b00-9c3a-1c28062e21a5 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.249804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014121e4-4ccc-4b00-9c3a-1c28062e21a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.264358 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-ca6e7fda-958d-478d-a19e-c51f2710ab43 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:10.265642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-ca6e7fda-958d-478d-a19e-c51f2710ab43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:10.265975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-014121e4-4ccc-4b00-9c3a-1c28062e21a5 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:10.267161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-014121e4-4ccc-4b00-9c3a-1c28062e21a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:10.272283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:10.326043 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 809/1609] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:10.326905 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e7fda-958d-478d-a19e-c51f2710ab43 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.331555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 801/1610] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:10.332975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014121e4-4ccc-4b00-9c3a-1c28062e21a5 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.336456 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e7fda-958d-478d-a19e-c51f2710ab43 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.342020 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b235cdf-1e1f-41d6-89b5-b88ed56e309c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.342226 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014121e4-4ccc-4b00-9c3a-1c28062e21a5 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.342914 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b235cdf-1e1f-41d6-89b5-b88ed56e309c admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.343200 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b235cdf-1e1f-41d6-89b5-b88ed56e309c admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.343488 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b235cdf-1e1f-41d6-89b5-b88ed56e309c admin admin] PATH_INFO: `/v3/projects/f03c2c0a06b64001b77bcef997e15a6e/users/ee2c89a61713457c9e3febf7ad85a1a9/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.345713 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b235cdf-1e1f-41d6-89b5-b88ed56e309c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f03c2c0a06b64001b77bcef997e15a6e, user_id=ee2c89a61713457c9e3febf7ad85a1a9, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.347599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769281f6-0542-456e-a831-1ea9fa443b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.348269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769281f6-0542-456e-a831-1ea9fa443b2b admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.348376 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b235cdf-1e1f-41d6-89b5-b88ed56e309c admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.348554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769281f6-0542-456e-a831-1ea9fa443b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.348858 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769281f6-0542-456e-a831-1ea9fa443b2b admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.349603 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769281f6-0542-456e-a831-1ea9fa443b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.351764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769281f6-0542-456e-a831-1ea9fa443b2b admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.359795 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 810/1611] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:10 2022] PUT /identity/v3/projects/f03c2c0a06b64001b77bcef997e15a6e/users/ee2c89a61713457c9e3febf7ad85a1a9/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:10.360575 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b235cdf-1e1f-41d6-89b5-b88ed56e309c admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.375276 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 802/1612] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:10 2022] POST /identity/v3/users => generated 296 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:10.376294 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769281f6-0542-456e-a831-1ea9fa443b2b admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.387260 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769281f6-0542-456e-a831-1ea9fa443b2b admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.430584 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b235cdf-1e1f-41d6-89b5-b88ed56e309c admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.435739 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7526a38-9e7c-4a42-882e-b8895ff8eca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.436626 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7526a38-9e7c-4a42-882e-b8895ff8eca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.436917 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7526a38-9e7c-4a42-882e-b8895ff8eca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.437220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7526a38-9e7c-4a42-882e-b8895ff8eca0 admin admin] PATH_INFO: `/v3/projects/461d19c7d1a4450eb4a9fd923dd13f7c/users/ad0d36d8ab824590a3c0c9ff92b2d7a9/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.444815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39325e-f9f0-46a0-8a11-199cca45ea3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.445671 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39325e-f9f0-46a0-8a11-199cca45ea3b admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.445952 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39325e-f9f0-46a0-8a11-199cca45ea3b admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.446246 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39325e-f9f0-46a0-8a11-199cca45ea3b admin admin] PATH_INFO: `/v3/projects/982c9543e7914eb0bbe4fb0fecba1d71/users/63c74334cc8f49148b2cd47245e021de/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.452073 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7526a38-9e7c-4a42-882e-b8895ff8eca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=461d19c7d1a4450eb4a9fd923dd13f7c, user_id=ad0d36d8ab824590a3c0c9ff92b2d7a9, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.454803 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7526a38-9e7c-4a42-882e-b8895ff8eca0 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.466051 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39325e-f9f0-46a0-8a11-199cca45ea3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=982c9543e7914eb0bbe4fb0fecba1d71, user_id=63c74334cc8f49148b2cd47245e021de, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.468446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 811/1613] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:10 2022] PUT /identity/v3/projects/461d19c7d1a4450eb4a9fd923dd13f7c/users/ad0d36d8ab824590a3c0c9ff92b2d7a9/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:10.468900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39325e-f9f0-46a0-8a11-199cca45ea3b admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.469423 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7526a38-9e7c-4a42-882e-b8895ff8eca0 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.489733 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:10.492746 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 803/1614] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:10 2022] PUT /identity/v3/projects/982c9543e7914eb0bbe4fb0fecba1d71/users/63c74334cc8f49148b2cd47245e021de/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:10.494070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddf540-892f-4784-91a7-94059a41c7f3 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.494350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddf540-892f-4784-91a7-94059a41c7f3 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.494619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddf540-892f-4784-91a7-94059a41c7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.512943 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-29ddf540-892f-4784-91a7-94059a41c7f3 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:10.514196 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-29ddf540-892f-4784-91a7-94059a41c7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:10.540346 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7526a38-9e7c-4a42-882e-b8895ff8eca0 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.575385 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 804/1615] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:10.576281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddf540-892f-4784-91a7-94059a41c7f3 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.577613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-29ddf540-892f-4784-91a7-94059a41c7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:10.598318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:10.600255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36e58f-27b7-41a0-b042-678315de98d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.601237 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36e58f-27b7-41a0-b042-678315de98d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.601519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36e58f-27b7-41a0-b042-678315de98d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.601792 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36e58f-27b7-41a0-b042-678315de98d1 admin admin] PATH_INFO: `/v3/projects/5eb82881a808418ebb4ea5bc87cf7e2c/users/34a9bfb0d0fc47b691948516e19075ab/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.617796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36e58f-27b7-41a0-b042-678315de98d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eb82881a808418ebb4ea5bc87cf7e2c, user_id=34a9bfb0d0fc47b691948516e19075ab, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.621139 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36e58f-27b7-41a0-b042-678315de98d1 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.636415 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 812/1616] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:10 2022] PUT /identity/v3/projects/5eb82881a808418ebb4ea5bc87cf7e2c/users/34a9bfb0d0fc47b691948516e19075ab/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:10.637362 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36e58f-27b7-41a0-b042-678315de98d1 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.642102 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddf540-892f-4784-91a7-94059a41c7f3 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.697560 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38c9a7-04e3-4cf4-8f2a-6fbdad095ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.698126 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38c9a7-04e3-4cf4-8f2a-6fbdad095ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.698442 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38c9a7-04e3-4cf4-8f2a-6fbdad095ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.698687 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38c9a7-04e3-4cf4-8f2a-6fbdad095ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.700063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36e58f-27b7-41a0-b042-678315de98d1 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.704218 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347b055-c955-4285-9aaf-22d54d8a1c45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.704862 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347b055-c955-4285-9aaf-22d54d8a1c45 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.705180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347b055-c955-4285-9aaf-22d54d8a1c45 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.705460 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347b055-c955-4285-9aaf-22d54d8a1c45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.759888 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38c9a7-04e3-4cf4-8f2a-6fbdad095ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.761412 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38c9a7-04e3-4cf4-8f2a-6fbdad095ee4 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.763724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:10.766386 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347b055-c955-4285-9aaf-22d54d8a1c45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.768801 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347b055-c955-4285-9aaf-22d54d8a1c45 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.771955 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 805/1617] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:10.773449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc49665-e2c6-40cb-b7dc-dae8d06c17ef None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.773746 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc49665-e2c6-40cb-b7dc-dae8d06c17ef None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.774045 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc49665-e2c6-40cb-b7dc-dae8d06c17ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.780724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 813/1618] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:10.781890 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347b055-c955-4285-9aaf-22d54d8a1c45 service neutron] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.791817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347b055-c955-4285-9aaf-22d54d8a1c45 service neutron] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.798568 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247396b-1b50-495d-979f-016ec33da4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.799540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247396b-1b50-495d-979f-016ec33da4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.799795 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247396b-1b50-495d-979f-016ec33da4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.800399 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247396b-1b50-495d-979f-016ec33da4b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.801778 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247396b-1b50-495d-979f-016ec33da4b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.803406 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247396b-1b50-495d-979f-016ec33da4b0 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.803677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-adc49665-e2c6-40cb-b7dc-dae8d06c17ef None None] MFA Rules not processed for user `ee2c89a61713457c9e3febf7ad85a1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:10.804922 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-adc49665-e2c6-40cb-b7dc-dae8d06c17ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:10.817132 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 814/1619] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:10.819185 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264b6a6-dd01-4a6b-8c05-a2d3d0a2ea07 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.819494 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264b6a6-dd01-4a6b-8c05-a2d3d0a2ea07 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.820145 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264b6a6-dd01-4a6b-8c05-a2d3d0a2ea07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.863067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-6264b6a6-dd01-4a6b-8c05-a2d3d0a2ea07 None None] MFA Rules not processed for user `ad0d36d8ab824590a3c0c9ff92b2d7a9`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:10.864451 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-6264b6a6-dd01-4a6b-8c05-a2d3d0a2ea07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:10.900982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 806/1620] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:10 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:10.904012 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf0155-e022-49e8-b327-15d37f3d3b1b None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.904562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf0155-e022-49e8-b327-15d37f3d3b1b None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.905870 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf0155-e022-49e8-b327-15d37f3d3b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.935312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-abaf0155-e022-49e8-b327-15d37f3d3b1b None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:10.937185 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-abaf0155-e022-49e8-b327-15d37f3d3b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:10.947281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 807/1621] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:10.949418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf0155-e022-49e8-b327-15d37f3d3b1b None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.951303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 815/1622] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:10 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:10.953376 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e370d-46e0-4a7f-996f-ccfc410c3870 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.954175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e370d-46e0-4a7f-996f-ccfc410c3870 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.954518 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e370d-46e0-4a7f-996f-ccfc410c3870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.960106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:10.968136 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf0155-e022-49e8-b327-15d37f3d3b1b None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.974351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8530a-0c9f-44ad-8282-0ba0da4e6cff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.975470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8530a-0c9f-44ad-8282-0ba0da4e6cff service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.976000 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8530a-0c9f-44ad-8282-0ba0da4e6cff service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.976347 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8530a-0c9f-44ad-8282-0ba0da4e6cff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.990576 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-3d9e370d-46e0-4a7f-996f-ccfc410c3870 None None] MFA Rules not processed for user `34a9bfb0d0fc47b691948516e19075ab`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:10.990795 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8530a-0c9f-44ad-8282-0ba0da4e6cff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.992149 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3d9e370d-46e0-4a7f-996f-ccfc410c3870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:10.992877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8530a-0c9f-44ad-8282-0ba0da4e6cff service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.007660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 808/1623] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:11.008911 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8530a-0c9f-44ad-8282-0ba0da4e6cff service neutron] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.030113 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8530a-0c9f-44ad-8282-0ba0da4e6cff service neutron] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:11.038128 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66ada8-3800-43f3-a45c-8775878f6df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:11.039267 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66ada8-3800-43f3-a45c-8775878f6df6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:11.039697 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66ada8-3800-43f3-a45c-8775878f6df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:11.041392 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66ada8-3800-43f3-a45c-8775878f6df6 admin admin] PATH_INFO: `/v3/projects/8e49e49760f344be884b94b54ee1c741/users/45cb29b5887d4c40b9cb1978d8ccdfa0/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:11.066459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66ada8-3800-43f3-a45c-8775878f6df6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e49e49760f344be884b94b54ee1c741, user_id=45cb29b5887d4c40b9cb1978d8ccdfa0, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:11.070566 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66ada8-3800-43f3-a45c-8775878f6df6 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.094068 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 809/1624] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:11 2022] PUT /identity/v3/projects/8e49e49760f344be884b94b54ee1c741/users/45cb29b5887d4c40b9cb1978d8ccdfa0/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:11.094068 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66ada8-3800-43f3-a45c-8775878f6df6 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.102005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 816/1625] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:11.106916 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e370d-46e0-4a7f-996f-ccfc410c3870 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.118921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e370d-46e0-4a7f-996f-ccfc410c3870 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:11.193757 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66ada8-3800-43f3-a45c-8775878f6df6 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:11.197881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e3c28-0888-4659-bf33-b824c10e2a91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:11.199075 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e3c28-0888-4659-bf33-b824c10e2a91 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:11.199867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e3c28-0888-4659-bf33-b824c10e2a91 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:11.199867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e3c28-0888-4659-bf33-b824c10e2a91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:11.218733 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8b2f9-b8a6-4153-94c1-e77b80327c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:11.220082 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8b2f9-b8a6-4153-94c1-e77b80327c29 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:11.220482 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8b2f9-b8a6-4153-94c1-e77b80327c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:11.221554 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8b2f9-b8a6-4153-94c1-e77b80327c29 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:11.222373 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8b2f9-b8a6-4153-94c1-e77b80327c29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:11.224483 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8b2f9-b8a6-4153-94c1-e77b80327c29 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.234543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:11.258180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 817/1626] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:11 2022] POST /identity/v3/projects => generated 357 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:11.259261 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8b2f9-b8a6-4153-94c1-e77b80327c29 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.316127 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449e3c28-0888-4659-bf33-b824c10e2a91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:11.317632 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449e3c28-0888-4659-bf33-b824c10e2a91 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.334548 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 810/1627] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:11 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:11.335754 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e3c28-0888-4659-bf33-b824c10e2a91 service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.353370 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e3c28-0888-4659-bf33-b824c10e2a91 service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:11.424652 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8b2f9-b8a6-4153-94c1-e77b80327c29 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:11.435382 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb0890-5505-4a6b-b00c-dc457c65811d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:11.435985 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb0890-5505-4a6b-b00c-dc457c65811d service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:11.436320 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb0890-5505-4a6b-b00c-dc457c65811d service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:11.436656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb0890-5505-4a6b-b00c-dc457c65811d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:11.505187 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832e7e6-80b6-47f1-aba3-ec573217cf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:11.505873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832e7e6-80b6-47f1-aba3-ec573217cf1a admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:11.506164 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832e7e6-80b6-47f1-aba3-ec573217cf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:11.506869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832e7e6-80b6-47f1-aba3-ec573217cf1a admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:11.507610 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832e7e6-80b6-47f1-aba3-ec573217cf1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:11.510391 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832e7e6-80b6-47f1-aba3-ec573217cf1a admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.527685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 811/1628] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:11.528603 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832e7e6-80b6-47f1-aba3-ec573217cf1a admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.546712 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:11.588609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb0890-5505-4a6b-b00c-dc457c65811d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:11.591202 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb0890-5505-4a6b-b00c-dc457c65811d service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.601893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832e7e6-80b6-47f1-aba3-ec573217cf1a admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:11.606402 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc89cc-f988-4e4a-ac19-786a90797bad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:11.607118 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc89cc-f988-4e4a-ac19-786a90797bad service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:11.607410 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc89cc-f988-4e4a-ac19-786a90797bad service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:11.607702 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc89cc-f988-4e4a-ac19-786a90797bad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:11.609482 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 818/1629] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:11.610809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb0890-5505-4a6b-b00c-dc457c65811d service neutron] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.625890 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb0890-5505-4a6b-b00c-dc457c65811d service neutron] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:11.635166 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb3493-f56e-4311-83e9-a10f5a418363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:11.636187 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb3493-f56e-4311-83e9-a10f5a418363 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:11.636784 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb3493-f56e-4311-83e9-a10f5a418363 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:11.637126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb3493-f56e-4311-83e9-a10f5a418363 admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:11.637981 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb3493-f56e-4311-83e9-a10f5a418363 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:11.641018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb3493-f56e-4311-83e9-a10f5a418363 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.681128 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 819/1630] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:11 2022] POST /identity/v3/users => generated 285 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:11.681478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb3493-f56e-4311-83e9-a10f5a418363 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.696454 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc89cc-f988-4e4a-ac19-786a90797bad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:11.698469 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc89cc-f988-4e4a-ac19-786a90797bad service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.701340 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb3493-f56e-4311-83e9-a10f5a418363 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:11.710684 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d0af5-ce12-4b9e-bba6-7cc65fbe57ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:11.711610 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d0af5-ce12-4b9e-bba6-7cc65fbe57ee service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:11.712021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d0af5-ce12-4b9e-bba6-7cc65fbe57ee service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:11.712395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d0af5-ce12-4b9e-bba6-7cc65fbe57ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:11.726259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 812/1631] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:11 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:11.729132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc89cc-f988-4e4a-ac19-786a90797bad service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.762815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc89cc-f988-4e4a-ac19-786a90797bad service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:11.775980 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd16ea-99fd-4be4-80e8-a697d2293fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:11.779510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd16ea-99fd-4be4-80e8-a697d2293fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:11.781981 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd16ea-99fd-4be4-80e8-a697d2293fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:11.782156 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd16ea-99fd-4be4-80e8-a697d2293fbd admin admin] PATH_INFO: `/v3/projects/8e49e49760f344be884b94b54ee1c741/users/45cb29b5887d4c40b9cb1978d8ccdfa0/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:11.790835 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd16ea-99fd-4be4-80e8-a697d2293fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e49e49760f344be884b94b54ee1c741, user_id=45cb29b5887d4c40b9cb1978d8ccdfa0, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:11.798676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd16ea-99fd-4be4-80e8-a697d2293fbd admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.824843 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d0af5-ce12-4b9e-bba6-7cc65fbe57ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:11.826658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d0af5-ce12-4b9e-bba6-7cc65fbe57ee service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.846207 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 813/1632] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:11 2022] PUT /identity/v3/projects/8e49e49760f344be884b94b54ee1c741/users/45cb29b5887d4c40b9cb1978d8ccdfa0/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:11.855560 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd16ea-99fd-4be4-80e8-a697d2293fbd admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.859425 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 820/1633] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:11.860936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ec54f-a019-4f91-a9cf-38df6606e51b None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:11.861257 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ec54f-a019-4f91-a9cf-38df6606e51b None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:11.861536 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ec54f-a019-4f91-a9cf-38df6606e51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:11.942290 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-685ec54f-a019-4f91-a9cf-38df6606e51b None None] MFA Rules not processed for user `45cb29b5887d4c40b9cb1978d8ccdfa0`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:11.943582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-685ec54f-a019-4f91-a9cf-38df6606e51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:11.978920 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:12.045405 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd16ea-99fd-4be4-80e8-a697d2293fbd admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:12.069061 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf8d77-e3eb-4a74-8899-8fc3aa923b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:12.070799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf8d77-e3eb-4a74-8899-8fc3aa923b62 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:12.074014 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf8d77-e3eb-4a74-8899-8fc3aa923b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:12.074833 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf8d77-e3eb-4a74-8899-8fc3aa923b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:12.076566 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf8d77-e3eb-4a74-8899-8fc3aa923b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:12.079712 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf8d77-e3eb-4a74-8899-8fc3aa923b62 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:12.109061 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 821/1634] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:12.110878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 814/1635] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:12.114665 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ec54f-a019-4f91-a9cf-38df6606e51b None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:12.136834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ec54f-a019-4f91-a9cf-38df6606e51b None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:12.150314 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53934a7-aa76-4a95-bd69-48196ce9a607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:12.153954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53934a7-aa76-4a95-bd69-48196ce9a607 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:12.154092 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53934a7-aa76-4a95-bd69-48196ce9a607 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:12.154298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53934a7-aa76-4a95-bd69-48196ce9a607 admin admin] PATH_INFO: `/v3/projects/0658ab86ba2c4fb8a8b24ace9aaa66b0/users/e6ac301f616f437b9f790757e2390fac/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:12.196634 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53934a7-aa76-4a95-bd69-48196ce9a607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0658ab86ba2c4fb8a8b24ace9aaa66b0, user_id=e6ac301f616f437b9f790757e2390fac, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:12.212685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53934a7-aa76-4a95-bd69-48196ce9a607 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:12.247985 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 822/1636] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:12 2022] PUT /identity/v3/projects/0658ab86ba2c4fb8a8b24ace9aaa66b0/users/e6ac301f616f437b9f790757e2390fac/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:12.248791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf8d77-e3eb-4a74-8899-8fc3aa923b62 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:12.267271 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53934a7-aa76-4a95-bd69-48196ce9a607 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:12.272377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf8d77-e3eb-4a74-8899-8fc3aa923b62 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:12.373733 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53934a7-aa76-4a95-bd69-48196ce9a607 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:12.379862 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8324f8-8b83-45b3-803a-721b3eeb3bbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:12.382116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8324f8-8b83-45b3-803a-721b3eeb3bbb service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:12.383694 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8324f8-8b83-45b3-803a-721b3eeb3bbb service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:12.384185 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8324f8-8b83-45b3-803a-721b3eeb3bbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:12.420238 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:12.450122 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6e94f-02e7-4d87-9f1e-0f8d4d1604fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:12.451075 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6e94f-02e7-4d87-9f1e-0f8d4d1604fb admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:12.453539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6e94f-02e7-4d87-9f1e-0f8d4d1604fb admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:12.454019 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6e94f-02e7-4d87-9f1e-0f8d4d1604fb admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:12.455869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6e94f-02e7-4d87-9f1e-0f8d4d1604fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:12.461019 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6e94f-02e7-4d87-9f1e-0f8d4d1604fb admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:12.491449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 815/1637] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:39:12.494570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6e94f-02e7-4d87-9f1e-0f8d4d1604fb admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:12.519210 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6e94f-02e7-4d87-9f1e-0f8d4d1604fb admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:12.534454 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8324f8-8b83-45b3-803a-721b3eeb3bbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:12.535080 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc38eff-0196-40c1-b2d9-e4d1199744f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:12.536041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8324f8-8b83-45b3-803a-721b3eeb3bbb service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:12.536616 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc38eff-0196-40c1-b2d9-e4d1199744f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:12.537161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc38eff-0196-40c1-b2d9-e4d1199744f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:12.537656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc38eff-0196-40c1-b2d9-e4d1199744f3 admin admin] PATH_INFO: `/v3/projects/0658ab86ba2c4fb8a8b24ace9aaa66b0/users/e6ac301f616f437b9f790757e2390fac/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:12.540865 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc38eff-0196-40c1-b2d9-e4d1199744f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0658ab86ba2c4fb8a8b24ace9aaa66b0, user_id=e6ac301f616f437b9f790757e2390fac, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:12.546817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc38eff-0196-40c1-b2d9-e4d1199744f3 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:12.565859 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 823/1638] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:39:12.577768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 816/1639] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:12 2022] PUT /identity/v3/projects/0658ab86ba2c4fb8a8b24ace9aaa66b0/users/e6ac301f616f437b9f790757e2390fac/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:12.588004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e841bc7-3fea-4ae4-a356-0ffc732b1837 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:12.588335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e841bc7-3fea-4ae4-a356-0ffc732b1837 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:12.588562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e841bc7-3fea-4ae4-a356-0ffc732b1837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:12.640173 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-2e841bc7-3fea-4ae4-a356-0ffc732b1837 None None] MFA Rules not processed for user `e6ac301f616f437b9f790757e2390fac`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:12.641451 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-2e841bc7-3fea-4ae4-a356-0ffc732b1837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:12.686109 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:12.827202 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 824/1640] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:12 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:13.031351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc38eff-0196-40c1-b2d9-e4d1199744f3 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:13.177824 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc38eff-0196-40c1-b2d9-e4d1199744f3 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:13.190154 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d700e-28e2-484b-b0ba-8a732f554dc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:13.191092 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d700e-28e2-484b-b0ba-8a732f554dc1 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:13.191092 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d700e-28e2-484b-b0ba-8a732f554dc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:13.191392 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d700e-28e2-484b-b0ba-8a732f554dc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:13.215590 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d700e-28e2-484b-b0ba-8a732f554dc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:13.217205 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d700e-28e2-484b-b0ba-8a732f554dc1 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:13.242764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 817/1641] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:13 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:13.321253 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e841bc7-3fea-4ae4-a356-0ffc732b1837 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:13.342660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e841bc7-3fea-4ae4-a356-0ffc732b1837 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:13.348951 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1e7c2-5bf5-430c-8e5d-e7a65df1a560 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:13.349932 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1e7c2-5bf5-430c-8e5d-e7a65df1a560 service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:13.350371 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1e7c2-5bf5-430c-8e5d-e7a65df1a560 service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:13.350843 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1e7c2-5bf5-430c-8e5d-e7a65df1a560 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:13.440358 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1e7c2-5bf5-430c-8e5d-e7a65df1a560 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:13.442481 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1e7c2-5bf5-430c-8e5d-e7a65df1a560 service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:13.455262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 825/1642] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:13.687328 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d700e-28e2-484b-b0ba-8a732f554dc1 service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:13.698864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d700e-28e2-484b-b0ba-8a732f554dc1 service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:13.711416 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0402fe2-4712-403c-b6ce-6ac28ab17b0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:13.712936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0402fe2-4712-403c-b6ce-6ac28ab17b0d service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:13.713467 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0402fe2-4712-403c-b6ce-6ac28ab17b0d service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:13.715465 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0402fe2-4712-403c-b6ce-6ac28ab17b0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:13.888540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0402fe2-4712-403c-b6ce-6ac28ab17b0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:13.890612 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0402fe2-4712-403c-b6ce-6ac28ab17b0d service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:13.918922 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 818/1643] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:13 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:14.519815 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69725f6d-47b7-4882-8414-12dfaf5ce0fd None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:14.520353 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69725f6d-47b7-4882-8414-12dfaf5ce0fd None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:14.520981 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69725f6d-47b7-4882-8414-12dfaf5ce0fd None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:14.523095 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 826/1644] 10.222.0.71 () {54 vars in 962 bytes} [Thu Jul 21 20:39:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:14.537416 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a2b42-8e18-4c1a-bf88-2f5839b4ea34 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:14.537918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a2b42-8e18-4c1a-bf88-2f5839b4ea34 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:14.538181 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a2b42-8e18-4c1a-bf88-2f5839b4ea34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:14.623673 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-e71a2b42-8e18-4c1a-bf88-2f5839b4ea34 None None] MFA Rules not processed for user `6fbc5240f5214aaea4418052b94739e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:14.625322 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-e71a2b42-8e18-4c1a-bf88-2f5839b4ea34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:14.744872 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:14.788095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 819/1645] 10.222.0.71 () {58 vars in 1059 bytes} [Thu Jul 21 20:39:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:14.789771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e13903-7c6b-4f62-af9b-88394dd033f7 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:14.789771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e13903-7c6b-4f62-af9b-88394dd033f7 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:14.789981 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e13903-7c6b-4f62-af9b-88394dd033f7 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:14.791248 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 827/1646] 10.222.0.71 () {54 vars in 912 bytes} [Thu Jul 21 20:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:14.801744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ab275-b52a-4cf7-9f09-b1f69efb37e6 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:14.802174 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ab275-b52a-4cf7-9f09-b1f69efb37e6 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:14.802551 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ab275-b52a-4cf7-9f09-b1f69efb37e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:14.853241 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-554ab275-b52a-4cf7-9f09-b1f69efb37e6 None None] MFA Rules not processed for user `6fbc5240f5214aaea4418052b94739e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:14.855117 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-554ab275-b52a-4cf7-9f09-b1f69efb37e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:14.876610 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 820/1647] 10.222.0.71 () {58 vars in 1009 bytes} [Thu Jul 21 20:39:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:14.892595 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e13903-7c6b-4f62-af9b-88394dd033f7 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:14.911229 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e13903-7c6b-4f62-af9b-88394dd033f7 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:14.918902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8c675-4a39-4c25-b03b-98c7fac42ad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:14.919580 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8c675-4a39-4c25-b03b-98c7fac42ad9 service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:14.919864 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8c675-4a39-4c25-b03b-98c7fac42ad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:14.920142 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8c675-4a39-4c25-b03b-98c7fac42ad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:15.051552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8c675-4a39-4c25-b03b-98c7fac42ad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:15.051552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8c675-4a39-4c25-b03b-98c7fac42ad9 service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:15.051552 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 828/1648] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:17.658516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5871f5c-8d8c-4389-9a89-783524252974 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:17.660304 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5871f5c-8d8c-4389-9a89-783524252974 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:17.660782 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5871f5c-8d8c-4389-9a89-783524252974 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:17.664110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 821/1649] 10.222.0.71 () {54 vars in 962 bytes} [Thu Jul 21 20:39:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:17.676605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15ebdb-bcdc-428f-af83-fe85ceb74791 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:17.677498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15ebdb-bcdc-428f-af83-fe85ceb74791 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:17.677917 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15ebdb-bcdc-428f-af83-fe85ceb74791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:17.744877 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-6d15ebdb-bcdc-428f-af83-fe85ceb74791 None None] MFA Rules not processed for user `6fbc5240f5214aaea4418052b94739e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:17.748382 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-6d15ebdb-bcdc-428f-af83-fe85ceb74791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:17.791450 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 829/1650] 10.222.0.71 () {58 vars in 1059 bytes} [Thu Jul 21 20:39:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:17.799705 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fd49a-0f25-48ec-9e74-db6d9c851fa9 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:17.799998 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fd49a-0f25-48ec-9e74-db6d9c851fa9 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:17.800282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fd49a-0f25-48ec-9e74-db6d9c851fa9 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:17.802234 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 822/1651] 10.222.0.71 () {54 vars in 912 bytes} [Thu Jul 21 20:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:17.807533 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64bb45-e329-4ad6-ae0c-ca250da6ac71 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:17.807851 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64bb45-e329-4ad6-ae0c-ca250da6ac71 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:17.808301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64bb45-e329-4ad6-ae0c-ca250da6ac71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:17.860682 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-9f64bb45-e329-4ad6-ae0c-ca250da6ac71 None None] MFA Rules not processed for user `6fbc5240f5214aaea4418052b94739e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:17.863045 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-9f64bb45-e329-4ad6-ae0c-ca250da6ac71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:17.890002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 830/1652] 10.222.0.71 () {58 vars in 1009 bytes} [Thu Jul 21 20:39:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:18.045409 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fd49a-0f25-48ec-9e74-db6d9c851fa9 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:18.060843 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fd49a-0f25-48ec-9e74-db6d9c851fa9 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:18.071352 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875c489-13ec-4c4d-9ad6-10027dcd9ba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:18.073475 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875c489-13ec-4c4d-9ad6-10027dcd9ba4 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:18.073905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875c489-13ec-4c4d-9ad6-10027dcd9ba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:18.074827 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875c489-13ec-4c4d-9ad6-10027dcd9ba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:18.114081 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c875c489-13ec-4c4d-9ad6-10027dcd9ba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:18.119713 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c875c489-13ec-4c4d-9ad6-10027dcd9ba4 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:18.135491 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 823/1653] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:19.764267 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473bc7f-080a-444c-a998-7b538ab5b97b None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:19.767853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473bc7f-080a-444c-a998-7b538ab5b97b None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:19.768555 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473bc7f-080a-444c-a998-7b538ab5b97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:19.826945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-f473bc7f-080a-444c-a998-7b538ab5b97b None None] MFA Rules not processed for user `5fc245eb98024dd382f2d2e72ee439de`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:19.828823 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f473bc7f-080a-444c-a998-7b538ab5b97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:19.975043 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 831/1654] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:19.991797 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875c489-13ec-4c4d-9ad6-10027dcd9ba4 service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:20.014941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875c489-13ec-4c4d-9ad6-10027dcd9ba4 service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:20.024533 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354755f-0e23-43a0-a33a-9d3dea1990a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:20.025269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354755f-0e23-43a0-a33a-9d3dea1990a9 service nova] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:20.025720 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354755f-0e23-43a0-a33a-9d3dea1990a9 service nova] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:20.026166 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354755f-0e23-43a0-a33a-9d3dea1990a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:20.044890 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354755f-0e23-43a0-a33a-9d3dea1990a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:20.046837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354755f-0e23-43a0-a33a-9d3dea1990a9 service nova] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:20.064367 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 824/1655] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:23.810907 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8882de-7d80-47dd-8a86-9b9bbc1a0064 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:23.811479 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8882de-7d80-47dd-8a86-9b9bbc1a0064 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:23.811572 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8882de-7d80-47dd-8a86-9b9bbc1a0064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:23.919629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-ba8882de-7d80-47dd-8a86-9b9bbc1a0064 None None] MFA Rules not processed for user `e971b95ceacc43d2ae82da35ec9f7fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:23.920862 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-ba8882de-7d80-47dd-8a86-9b9bbc1a0064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:23.938778 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:24.133385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 832/1656] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:24.145429 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da863c19-634f-4a6f-b4da-e36e0d9b90e0 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:24.146329 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da863c19-634f-4a6f-b4da-e36e0d9b90e0 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:24.146818 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da863c19-634f-4a6f-b4da-e36e0d9b90e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:24.194294 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-da863c19-634f-4a6f-b4da-e36e0d9b90e0 None None] MFA Rules not processed for user `e971b95ceacc43d2ae82da35ec9f7fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:24.197537 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-da863c19-634f-4a6f-b4da-e36e0d9b90e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:24.243767 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 825/1657] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:24.253864 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8882de-7d80-47dd-8a86-9b9bbc1a0064 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:24.289605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8882de-7d80-47dd-8a86-9b9bbc1a0064 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:24.310408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf1ab8-fbcb-4e2d-af11-40a0116d58f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:24.311083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf1ab8-fbcb-4e2d-af11-40a0116d58f8 service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:24.311367 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf1ab8-fbcb-4e2d-af11-40a0116d58f8 service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:24.311660 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf1ab8-fbcb-4e2d-af11-40a0116d58f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:24.335430 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf1ab8-fbcb-4e2d-af11-40a0116d58f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:24.337107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf1ab8-fbcb-4e2d-af11-40a0116d58f8 service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:24.381554 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 833/1658] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:26.419974 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7305862-4984-4046-8f4d-7765ebee2b55 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:26.420529 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7305862-4984-4046-8f4d-7765ebee2b55 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:26.420773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7305862-4984-4046-8f4d-7765ebee2b55 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:26.423701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 826/1659] 10.222.0.71 () {54 vars in 912 bytes} [Thu Jul 21 20:39:26 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:26.429461 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5074437-2abc-43d1-a573-620da37d5e51 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:26.430573 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5074437-2abc-43d1-a573-620da37d5e51 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:26.430998 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5074437-2abc-43d1-a573-620da37d5e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:26.561037 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-d5074437-2abc-43d1-a573-620da37d5e51 None None] MFA Rules not processed for user `03eee8ef92314296b7d5c14c52f0d080`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:26.568356 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-d5074437-2abc-43d1-a573-620da37d5e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:26.613699 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137e5bb-9889-4e15-8b29-877459b146af None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:26.614363 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137e5bb-9889-4e15-8b29-877459b146af None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:26.614755 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137e5bb-9889-4e15-8b29-877459b146af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:26.729541 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-e137e5bb-9889-4e15-8b29-877459b146af None None] MFA Rules not processed for user `63c74334cc8f49148b2cd47245e021de`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:26.730800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-e137e5bb-9889-4e15-8b29-877459b146af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:26.810579 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 834/1660] 10.222.0.71 () {58 vars in 1009 bytes} [Thu Jul 21 20:39:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:26.848659 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5074437-2abc-43d1-a573-620da37d5e51 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:26.862776 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5074437-2abc-43d1-a573-620da37d5e51 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:26.889877 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0994c5f-5182-4132-9db0-98c4ff8ef1a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:26.890658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0994c5f-5182-4132-9db0-98c4ff8ef1a2 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:26.890975 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0994c5f-5182-4132-9db0-98c4ff8ef1a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:26.891281 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0994c5f-5182-4132-9db0-98c4ff8ef1a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:26.923716 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 827/1661] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:26.927165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee6807-063a-4125-a3f2-032c83ee559b None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:26.929938 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee6807-063a-4125-a3f2-032c83ee559b None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:26.930806 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee6807-063a-4125-a3f2-032c83ee559b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:26.940592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0994c5f-5182-4132-9db0-98c4ff8ef1a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:26.942540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0994c5f-5182-4132-9db0-98c4ff8ef1a2 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:26.986161 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 835/1662] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:39:26.987237 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0994c5f-5182-4132-9db0-98c4ff8ef1a2 service neutron] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:27.050221 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0994c5f-5182-4132-9db0-98c4ff8ef1a2 service neutron] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:27.071109 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8713ff-e72c-4cb4-9b77-4d2fc2509fdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:27.072085 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8713ff-e72c-4cb4-9b77-4d2fc2509fdf service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:27.072581 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8713ff-e72c-4cb4-9b77-4d2fc2509fdf service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:27.073014 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8713ff-e72c-4cb4-9b77-4d2fc2509fdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:27.075084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-12ee6807-063a-4125-a3f2-032c83ee559b None None] MFA Rules not processed for user `654b47930e6f4a8ca378d6876cc8ae6c`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:27.076603 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-12ee6807-063a-4125-a3f2-032c83ee559b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:27.095154 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8713ff-e72c-4cb4-9b77-4d2fc2509fdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:27.097051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8713ff-e72c-4cb4-9b77-4d2fc2509fdf service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:27.124148 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 836/1663] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:27.292368 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 828/1664] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:27.304115 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae203ba-0a9e-4fae-b207-ea0f1cc2d4b2 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:27.304698 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae203ba-0a9e-4fae-b207-ea0f1cc2d4b2 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:27.305220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae203ba-0a9e-4fae-b207-ea0f1cc2d4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:27.396112 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-5ae203ba-0a9e-4fae-b207-ea0f1cc2d4b2 None None] MFA Rules not processed for user `654b47930e6f4a8ca378d6876cc8ae6c`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:27.397639 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-5ae203ba-0a9e-4fae-b207-ea0f1cc2d4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:27.445638 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 837/1665] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:27.470193 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee6807-063a-4125-a3f2-032c83ee559b None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:27.501132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee6807-063a-4125-a3f2-032c83ee559b None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:27.527631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc373c2-0bb6-49f5-989f-fba4682ef0e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:27.528531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc373c2-0bb6-49f5-989f-fba4682ef0e2 service nova] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:27.528972 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc373c2-0bb6-49f5-989f-fba4682ef0e2 service nova] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:27.529485 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc373c2-0bb6-49f5-989f-fba4682ef0e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:27.566097 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc373c2-0bb6-49f5-989f-fba4682ef0e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:27.567728 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc373c2-0bb6-49f5-989f-fba4682ef0e2 service nova] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:27.610489 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 829/1666] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:28.360404 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64591f-0037-4613-8e55-e858d27a0402 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:28.360725 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64591f-0037-4613-8e55-e858d27a0402 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:28.361037 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64591f-0037-4613-8e55-e858d27a0402 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:28.362891 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 838/1667] 10.222.0.71 () {54 vars in 961 bytes} [Thu Jul 21 20:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:28.377445 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08063622-ffc6-4077-9451-1044b1028006 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:28.378236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08063622-ffc6-4077-9451-1044b1028006 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:28.378882 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08063622-ffc6-4077-9451-1044b1028006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:28.425789 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-08063622-ffc6-4077-9451-1044b1028006 None None] MFA Rules not processed for user `364e80ec79b94812a1df76c62c2b13a0`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:28.427536 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-08063622-ffc6-4077-9451-1044b1028006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:28.483715 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:28.523055 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829983e4-f3c0-490e-870d-5065676b4e4b None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:28.523614 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829983e4-f3c0-490e-870d-5065676b4e4b None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:28.523614 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829983e4-f3c0-490e-870d-5065676b4e4b None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:28.525725 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 839/1668] 10.222.0.71 () {54 vars in 912 bytes} [Thu Jul 21 20:39:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:28.552446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d670278-9bb9-487c-b5ea-296792310073 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:28.553251 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d670278-9bb9-487c-b5ea-296792310073 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:28.553874 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d670278-9bb9-487c-b5ea-296792310073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:28.591479 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-1d670278-9bb9-487c-b5ea-296792310073 None None] MFA Rules not processed for user `03eee8ef92314296b7d5c14c52f0d080`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:28.593248 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-1d670278-9bb9-487c-b5ea-296792310073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:28.608753 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 840/1669] 10.222.0.71 () {58 vars in 1009 bytes} [Thu Jul 21 20:39:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:28.610423 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541b02a-7fba-4612-8eda-65651d2c0076 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:28.610718 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541b02a-7fba-4612-8eda-65651d2c0076 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:28.610969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541b02a-7fba-4612-8eda-65651d2c0076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:28.662439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-d541b02a-7fba-4612-8eda-65651d2c0076 None None] MFA Rules not processed for user `5fc245eb98024dd382f2d2e72ee439de`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:28.666992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-d541b02a-7fba-4612-8eda-65651d2c0076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:28.688116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 841/1670] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:28.689006 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541b02a-7fba-4612-8eda-65651d2c0076 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:28.690264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 830/1671] 10.222.0.71 () {58 vars in 1058 bytes} [Thu Jul 21 20:39:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:28.703290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08063622-ffc6-4077-9451-1044b1028006 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:28.721871 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541b02a-7fba-4612-8eda-65651d2c0076 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:28.722577 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08063622-ffc6-4077-9451-1044b1028006 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:28.730733 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7dac5-cdff-4de3-b245-9d6db5ba1631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:28.731949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7dac5-cdff-4de3-b245-9d6db5ba1631 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:28.732856 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7dac5-cdff-4de3-b245-9d6db5ba1631 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:28.735174 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7dac5-cdff-4de3-b245-9d6db5ba1631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:28.737865 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4bf86-34a2-4bae-a5dd-04a2b9b5e375 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '364e80ec79b94812a1df76c62c2b13a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:28.741134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4bf86-34a2-4bae-a5dd-04a2b9b5e375 service placement] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:28.741491 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4bf86-34a2-4bae-a5dd-04a2b9b5e375 service placement] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:28.741775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4bf86-34a2-4bae-a5dd-04a2b9b5e375 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:28.747120 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-b1b4bf86-34a2-4bae-a5dd-04a2b9b5e375 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:28.759105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7dac5-cdff-4de3-b245-9d6db5ba1631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:28.760566 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7dac5-cdff-4de3-b245-9d6db5ba1631 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:28.793443 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4bf86-34a2-4bae-a5dd-04a2b9b5e375 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:28.805429 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4bf86-34a2-4bae-a5dd-04a2b9b5e375 service placement] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:28.810732 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 842/1672] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:28.818018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7dac5-cdff-4de3-b245-9d6db5ba1631 service neutron] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:28.830924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 831/1673] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:28.903842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:29.008507 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7dac5-cdff-4de3-b245-9d6db5ba1631 service neutron] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:29.015148 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab00cf5-7cf5-4f88-8795-335999a77735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:29.023711 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab00cf5-7cf5-4f88-8795-335999a77735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:29.024182 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab00cf5-7cf5-4f88-8795-335999a77735 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:29.025820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab00cf5-7cf5-4f88-8795-335999a77735 admin admin] PATH_INFO: `/v3/users/66d711e34087402abd12dae6113f6053` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:29.033698 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab00cf5-7cf5-4f88-8795-335999a77735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66d711e34087402abd12dae6113f6053)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:29.035042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: WARNING py.warnings [None req-dab00cf5-7cf5-4f88-8795-335999a77735 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:39:29.035042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: warnings.warn(msg) Jul 21 20:39:29.035042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: Jul 21 20:39:29.037694 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab00cf5-7cf5-4f88-8795-335999a77735 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:29.188223 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-dab00cf5-7cf5-4f88-8795-335999a77735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66d711e34087402abd12dae6113f6053'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:29.206049 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-dab00cf5-7cf5-4f88-8795-335999a77735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66d711e34087402abd12dae6113f6053'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:29.206901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-dab00cf5-7cf5-4f88-8795-335999a77735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66d711e34087402abd12dae6113f6053'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:29.230173 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-dab00cf5-7cf5-4f88-8795-335999a77735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66d711e34087402abd12dae6113f6053'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:29.230924 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-dab00cf5-7cf5-4f88-8795-335999a77735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66d711e34087402abd12dae6113f6053'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:29.293574 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 843/1674] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:39:28 2022] DELETE /identity/v3/users/66d711e34087402abd12dae6113f6053 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:29.295548 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052fc15-81da-4a80-94d7-6d4d1af1d18e None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:29.296128 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052fc15-81da-4a80-94d7-6d4d1af1d18e None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:29.296555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052fc15-81da-4a80-94d7-6d4d1af1d18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:29.338160 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-0052fc15-81da-4a80-94d7-6d4d1af1d18e None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:29.339738 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-0052fc15-81da-4a80-94d7-6d4d1af1d18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:29.490129 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 832/1675] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:29.505607 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab00cf5-7cf5-4f88-8795-335999a77735 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:29.507498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-dab00cf5-7cf5-4f88-8795-335999a77735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:29.588067 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052fc15-81da-4a80-94d7-6d4d1af1d18e None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:29.593100 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-0052fc15-81da-4a80-94d7-6d4d1af1d18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:29.627772 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052fc15-81da-4a80-94d7-6d4d1af1d18e None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:29.647362 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e007b-1c1d-4457-b43c-1b83a72ecad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:29.649027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e007b-1c1d-4457-b43c-1b83a72ecad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:29.653349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e007b-1c1d-4457-b43c-1b83a72ecad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:29.654127 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e007b-1c1d-4457-b43c-1b83a72ecad2 admin admin] PATH_INFO: `/v3/users/34a9bfb0d0fc47b691948516e19075ab` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:29.659323 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e007b-1c1d-4457-b43c-1b83a72ecad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34a9bfb0d0fc47b691948516e19075ab)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:29.662093 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab00cf5-7cf5-4f88-8795-335999a77735 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:29.672304 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e007b-1c1d-4457-b43c-1b83a72ecad2 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:29.674931 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4accf5e4-8d3f-41c3-856f-5d954feada56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:29.675611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accf5e4-8d3f-41c3-856f-5d954feada56 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:29.675893 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accf5e4-8d3f-41c3-856f-5d954feada56 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:29.678337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accf5e4-8d3f-41c3-856f-5d954feada56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:29.705280 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4accf5e4-8d3f-41c3-856f-5d954feada56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:29.709699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4accf5e4-8d3f-41c3-856f-5d954feada56 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:29.745717 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 844/1676] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:29.849780 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f69e007b-1c1d-4457-b43c-1b83a72ecad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34a9bfb0d0fc47b691948516e19075ab'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:29.870873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f69e007b-1c1d-4457-b43c-1b83a72ecad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34a9bfb0d0fc47b691948516e19075ab'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:29.873505 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f69e007b-1c1d-4457-b43c-1b83a72ecad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34a9bfb0d0fc47b691948516e19075ab'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:29.902592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f69e007b-1c1d-4457-b43c-1b83a72ecad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34a9bfb0d0fc47b691948516e19075ab'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:29.918486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f69e007b-1c1d-4457-b43c-1b83a72ecad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34a9bfb0d0fc47b691948516e19075ab'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:29.943089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 833/1677] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:39:29 2022] DELETE /identity/v3/users/34a9bfb0d0fc47b691948516e19075ab => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:29.947661 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d70ad-730b-4a7d-b124-c090440b787a None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:29.949172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d70ad-730b-4a7d-b124-c090440b787a None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:29.949616 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d70ad-730b-4a7d-b124-c090440b787a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:29.980070 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-9b5d70ad-730b-4a7d-b124-c090440b787a None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:29.987747 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-9b5d70ad-730b-4a7d-b124-c090440b787a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:30.118247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:30.159404 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 845/1678] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:30.225969 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e007b-1c1d-4457-b43c-1b83a72ecad2 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:30.227701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-f69e007b-1c1d-4457-b43c-1b83a72ecad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:30.257435 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:30.406268 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e007b-1c1d-4457-b43c-1b83a72ecad2 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:30.422598 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76d078-2a01-4baf-8e54-4deaacc16f83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:30.423568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76d078-2a01-4baf-8e54-4deaacc16f83 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:30.423568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76d078-2a01-4baf-8e54-4deaacc16f83 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:30.423795 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76d078-2a01-4baf-8e54-4deaacc16f83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:30.447295 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76d078-2a01-4baf-8e54-4deaacc16f83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:30.456959 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76d078-2a01-4baf-8e54-4deaacc16f83 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:30.483179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 834/1679] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:30.723945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b3ffc-cbe6-4400-bf95-4e42f4db38a4 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:30.723945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b3ffc-cbe6-4400-bf95-4e42f4db38a4 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:30.723945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b3ffc-cbe6-4400-bf95-4e42f4db38a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:30.781707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-ae5b3ffc-cbe6-4400-bf95-4e42f4db38a4 None None] MFA Rules not processed for user `654b47930e6f4a8ca378d6876cc8ae6c`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:30.783250 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-ae5b3ffc-cbe6-4400-bf95-4e42f4db38a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:30.936207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 846/1680] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:31.571143 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76d078-2a01-4baf-8e54-4deaacc16f83 service neutron] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:31.572780 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-ba76d078-2a01-4baf-8e54-4deaacc16f83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:31.643242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76d078-2a01-4baf-8e54-4deaacc16f83 service neutron] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:31.675903 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b732d-4659-43b8-a35f-359c57c58c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:31.679898 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b732d-4659-43b8-a35f-359c57c58c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:31.680278 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b732d-4659-43b8-a35f-359c57c58c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:31.681739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b732d-4659-43b8-a35f-359c57c58c16 admin admin] PATH_INFO: `/v3/projects/620b43f77db64f51a974bce26c1417c3` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:31.688497 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b732d-4659-43b8-a35f-359c57c58c16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=620b43f77db64f51a974bce26c1417c3)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:31.693502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: WARNING py.warnings [None req-671b732d-4659-43b8-a35f-359c57c58c16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:39:31.693502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: warnings.warn(msg) Jul 21 20:39:31.693502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: Jul 21 20:39:31.698332 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b732d-4659-43b8-a35f-359c57c58c16 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:31.949975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-671b732d-4659-43b8-a35f-359c57c58c16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '620b43f77db64f51a974bce26c1417c3', 'request_id': 'req-671b732d-4659-43b8-a35f-359c57c58c16'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:31.982133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-671b732d-4659-43b8-a35f-359c57c58c16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '620b43f77db64f51a974bce26c1417c3', 'request_id': 'req-671b732d-4659-43b8-a35f-359c57c58c16'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:32.014037 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-671b732d-4659-43b8-a35f-359c57c58c16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '620b43f77db64f51a974bce26c1417c3', 'request_id': 'req-671b732d-4659-43b8-a35f-359c57c58c16'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:32.078298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-671b732d-4659-43b8-a35f-359c57c58c16 admin admin] The token cache is being invalidate because project 620b43f77db64f51a974bce26c1417c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:39:32.078631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-671b732d-4659-43b8-a35f-359c57c58c16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:32.083419 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-671b732d-4659-43b8-a35f-359c57c58c16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:32.087626 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 835/1681] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:39:31 2022] DELETE /identity/v3/projects/620b43f77db64f51a974bce26c1417c3 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:32.092387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b3ffc-cbe6-4400-bf95-4e42f4db38a4 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:32.117385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b3ffc-cbe6-4400-bf95-4e42f4db38a4 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:32.370931 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d2f6c-e7ef-49c4-a6ba-a10c690c46db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:32.372152 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d2f6c-e7ef-49c4-a6ba-a10c690c46db admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:32.372586 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d2f6c-e7ef-49c4-a6ba-a10c690c46db admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:32.373347 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d2f6c-e7ef-49c4-a6ba-a10c690c46db admin admin] PATH_INFO: `/v3/users/45cb29b5887d4c40b9cb1978d8ccdfa0` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:32.375112 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d2f6c-e7ef-49c4-a6ba-a10c690c46db admin admin] RBAC: Authorizing `identity:delete_user(user_id=45cb29b5887d4c40b9cb1978d8ccdfa0)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:32.381166 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d2f6c-e7ef-49c4-a6ba-a10c690c46db admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:32.530928 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-122d2f6c-e7ef-49c4-a6ba-a10c690c46db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45cb29b5887d4c40b9cb1978d8ccdfa0'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:32.560642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-122d2f6c-e7ef-49c4-a6ba-a10c690c46db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45cb29b5887d4c40b9cb1978d8ccdfa0'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:32.561758 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-122d2f6c-e7ef-49c4-a6ba-a10c690c46db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45cb29b5887d4c40b9cb1978d8ccdfa0'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:32.590328 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-122d2f6c-e7ef-49c4-a6ba-a10c690c46db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45cb29b5887d4c40b9cb1978d8ccdfa0'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:32.593367 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-122d2f6c-e7ef-49c4-a6ba-a10c690c46db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45cb29b5887d4c40b9cb1978d8ccdfa0'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:32.678014 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 847/1682] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:39:32 2022] DELETE /identity/v3/users/45cb29b5887d4c40b9cb1978d8ccdfa0 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:39:32.716220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b732d-4659-43b8-a35f-359c57c58c16 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:32.718010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-671b732d-4659-43b8-a35f-359c57c58c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:32.762428 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b732d-4659-43b8-a35f-359c57c58c16 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:32.873516 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322db15-880b-4388-8dfd-450ec9861f59 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:32.874105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322db15-880b-4388-8dfd-450ec9861f59 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:32.874284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322db15-880b-4388-8dfd-450ec9861f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:32.957812 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:33.041725 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f4b0f-d0c2-46ed-9056-a07f4463e42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:33.050850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f4b0f-d0c2-46ed-9056-a07f4463e42e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:33.052516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f4b0f-d0c2-46ed-9056-a07f4463e42e admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:33.056093 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f4b0f-d0c2-46ed-9056-a07f4463e42e admin admin] PATH_INFO: `/v3/projects/5eb82881a808418ebb4ea5bc87cf7e2c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:33.060061 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f4b0f-d0c2-46ed-9056-a07f4463e42e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eb82881a808418ebb4ea5bc87cf7e2c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:33.074040 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f4b0f-d0c2-46ed-9056-a07f4463e42e admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:33.074726 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-3322db15-880b-4388-8dfd-450ec9861f59 None None] MFA Rules not processed for user `b94e634e8855486a9ce14d3de9d986fb`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:33.079839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3322db15-880b-4388-8dfd-450ec9861f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:33.250807 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:33.273689 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-da0f4b0f-d0c2-46ed-9056-a07f4463e42e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eb82881a808418ebb4ea5bc87cf7e2c', 'request_id': 'req-da0f4b0f-d0c2-46ed-9056-a07f4463e42e'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:33.294983 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 848/1683] 10.222.0.71 () {58 vars in 1009 bytes} [Thu Jul 21 20:39:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:39:33.302232 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-da0f4b0f-d0c2-46ed-9056-a07f4463e42e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eb82881a808418ebb4ea5bc87cf7e2c', 'request_id': 'req-da0f4b0f-d0c2-46ed-9056-a07f4463e42e'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:33.350448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-da0f4b0f-d0c2-46ed-9056-a07f4463e42e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eb82881a808418ebb4ea5bc87cf7e2c', 'request_id': 'req-da0f4b0f-d0c2-46ed-9056-a07f4463e42e'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:33.442677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-da0f4b0f-d0c2-46ed-9056-a07f4463e42e admin admin] The token cache is being invalidate because project 5eb82881a808418ebb4ea5bc87cf7e2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:39:33.443182 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-da0f4b0f-d0c2-46ed-9056-a07f4463e42e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:33.446203 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-da0f4b0f-d0c2-46ed-9056-a07f4463e42e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:33.455593 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cfc230-6f83-4cdd-9d76-bbd260472bb1 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:33.455899 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cfc230-6f83-4cdd-9d76-bbd260472bb1 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:33.456152 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cfc230-6f83-4cdd-9d76-bbd260472bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:33.458936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 836/1684] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:39:32 2022] DELETE /identity/v3/projects/5eb82881a808418ebb4ea5bc87cf7e2c => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:33.472812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1e151-59ed-44a5-90b3-614e5c622abe None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:33.476372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1e151-59ed-44a5-90b3-614e5c622abe None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:33.476686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1e151-59ed-44a5-90b3-614e5c622abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:33.504536 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-f6cfc230-6f83-4cdd-9d76-bbd260472bb1 None None] MFA Rules not processed for user `63c74334cc8f49148b2cd47245e021de`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:33.511334 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f6cfc230-6f83-4cdd-9d76-bbd260472bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:33.572577 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-d5e1e151-59ed-44a5-90b3-614e5c622abe None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:33.587710 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d5e1e151-59ed-44a5-90b3-614e5c622abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:33.827885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 849/1685] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:33.828810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cfc230-6f83-4cdd-9d76-bbd260472bb1 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:33.835523 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f6cfc230-6f83-4cdd-9d76-bbd260472bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:33.889894 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 837/1686] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:33.900561 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1e151-59ed-44a5-90b3-614e5c622abe None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:33.905912 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d5e1e151-59ed-44a5-90b3-614e5c622abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:33.939324 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1e151-59ed-44a5-90b3-614e5c622abe None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:33.959211 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80aef8-0891-4325-9a23-a836b937d284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:33.963314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80aef8-0891-4325-9a23-a836b937d284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:33.964406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80aef8-0891-4325-9a23-a836b937d284 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:33.965606 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80aef8-0891-4325-9a23-a836b937d284 admin admin] PATH_INFO: `/v3/projects/8e49e49760f344be884b94b54ee1c741` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:33.971883 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80aef8-0891-4325-9a23-a836b937d284 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e49e49760f344be884b94b54ee1c741)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:33.981967 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80aef8-0891-4325-9a23-a836b937d284 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:34.142558 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-9b80aef8-0891-4325-9a23-a836b937d284 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e49e49760f344be884b94b54ee1c741', 'request_id': 'req-9b80aef8-0891-4325-9a23-a836b937d284'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:34.156408 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-9b80aef8-0891-4325-9a23-a836b937d284 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e49e49760f344be884b94b54ee1c741', 'request_id': 'req-9b80aef8-0891-4325-9a23-a836b937d284'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:34.164736 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-9b80aef8-0891-4325-9a23-a836b937d284 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e49e49760f344be884b94b54ee1c741', 'request_id': 'req-9b80aef8-0891-4325-9a23-a836b937d284'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:34.179610 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-9b80aef8-0891-4325-9a23-a836b937d284 admin admin] The token cache is being invalidate because project 8e49e49760f344be884b94b54ee1c741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:39:34.180213 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-9b80aef8-0891-4325-9a23-a836b937d284 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:34.181045 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-9b80aef8-0891-4325-9a23-a836b937d284 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:34.188566 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 838/1687] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:39:33 2022] DELETE /identity/v3/projects/8e49e49760f344be884b94b54ee1c741 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:34.190399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f267d61-7195-48c0-8de6-5ae4940c4dfa None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:34.190891 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f267d61-7195-48c0-8de6-5ae4940c4dfa None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:34.191326 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f267d61-7195-48c0-8de6-5ae4940c4dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:34.218008 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-2f267d61-7195-48c0-8de6-5ae4940c4dfa None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:34.220035 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-2f267d61-7195-48c0-8de6-5ae4940c4dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:34.222370 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cfc230-6f83-4cdd-9d76-bbd260472bb1 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:34.267054 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:34.309465 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:34.332181 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 839/1688] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:34.334398 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa595f1-8cbc-4ed0-be9d-90b66938312f None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:34.334852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa595f1-8cbc-4ed0-be9d-90b66938312f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:34.335787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa595f1-8cbc-4ed0-be9d-90b66938312f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:34.425737 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-4fa595f1-8cbc-4ed0-be9d-90b66938312f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:34.432110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-4fa595f1-8cbc-4ed0-be9d-90b66938312f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:34.465154 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 840/1689] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:34.466480 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa595f1-8cbc-4ed0-be9d-90b66938312f None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:34.471310 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500085ec-0cfb-49bd-9b7c-4da9bad5316b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3babb078203647e2b3ef3b2787b84e3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:34.472676 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500085ec-0cfb-49bd-9b7c-4da9bad5316b service swift] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:34.472967 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500085ec-0cfb-49bd-9b7c-4da9bad5316b service swift] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:34.473269 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500085ec-0cfb-49bd-9b7c-4da9bad5316b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:34.476991 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-500085ec-0cfb-49bd-9b7c-4da9bad5316b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:34.492100 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa595f1-8cbc-4ed0-be9d-90b66938312f None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:34.514598 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcc89b-d8fa-4f0c-a2d3-cd48d5a50486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:34.518221 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcc89b-d8fa-4f0c-a2d3-cd48d5a50486 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:34.519202 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcc89b-d8fa-4f0c-a2d3-cd48d5a50486 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:34.520707 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcc89b-d8fa-4f0c-a2d3-cd48d5a50486 admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:34.525337 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcc89b-d8fa-4f0c-a2d3-cd48d5a50486 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:34.525688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcc89b-d8fa-4f0c-a2d3-cd48d5a50486 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:34.529741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcc89b-d8fa-4f0c-a2d3-cd48d5a50486 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:34.573579 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 841/1690] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:34.577722 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3a4f7-a419-44f8-b76d-f3b7c487c825 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:34.578015 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3a4f7-a419-44f8-b76d-f3b7c487c825 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:34.578294 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3a4f7-a419-44f8-b76d-f3b7c487c825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:34.625332 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-20a3a4f7-a419-44f8-b76d-f3b7c487c825 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:34.629631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-20a3a4f7-a419-44f8-b76d-f3b7c487c825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:34.656755 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 842/1691] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:34.658304 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3a4f7-a419-44f8-b76d-f3b7c487c825 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:34.674606 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3a4f7-a419-44f8-b76d-f3b7c487c825 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:34.674812 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500085ec-0cfb-49bd-9b7c-4da9bad5316b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:34.676385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500085ec-0cfb-49bd-9b7c-4da9bad5316b service swift] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:34.683315 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbe1d8-4fd4-4d41-ac01-c3594c57ce46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:34.692408 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbe1d8-4fd4-4d41-ac01-c3594c57ce46 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:34.692408 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbe1d8-4fd4-4d41-ac01-c3594c57ce46 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:34.692408 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbe1d8-4fd4-4d41-ac01-c3594c57ce46 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:34.692408 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbe1d8-4fd4-4d41-ac01-c3594c57ce46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:34.692408 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbe1d8-4fd4-4d41-ac01-c3594c57ce46 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:34.698717 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 850/1692] 10.222.0.71 () {60 vars in 1384 bytes} [Thu Jul 21 20:39:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 869 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 20:39:34.699782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500085ec-0cfb-49bd-9b7c-4da9bad5316b service swift] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:34.741375 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500085ec-0cfb-49bd-9b7c-4da9bad5316b service swift] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:34.743093 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 843/1693] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:34 2022] POST /identity/v3/projects => generated 403 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:34.744199 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbe1d8-4fd4-4d41-ac01-c3594c57ce46 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:34.891109 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbe1d8-4fd4-4d41-ac01-c3594c57ce46 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:34.903892 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7fa7d-1841-43b0-8cf0-44754f474ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:34.906852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7fa7d-1841-43b0-8cf0-44754f474ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:34.907166 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7fa7d-1841-43b0-8cf0-44754f474ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:34.908777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7fa7d-1841-43b0-8cf0-44754f474ad9 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:34.910809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7fa7d-1841-43b0-8cf0-44754f474ad9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:34.916409 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7fa7d-1841-43b0-8cf0-44754f474ad9 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:34.978950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfa701-74b4-4879-92a8-7d5ef3cdf524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:34.980481 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfa701-74b4-4879-92a8-7d5ef3cdf524 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:34.980670 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 844/1694] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:34 2022] POST /identity/v3/users => generated 302 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:34.981697 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7fa7d-1841-43b0-8cf0-44754f474ad9 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:34.982291 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfa701-74b4-4879-92a8-7d5ef3cdf524 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:34.983390 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfa701-74b4-4879-92a8-7d5ef3cdf524 admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:34.993106 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfa701-74b4-4879-92a8-7d5ef3cdf524 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:34.994903 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfa701-74b4-4879-92a8-7d5ef3cdf524 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:35.000169 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfa701-74b4-4879-92a8-7d5ef3cdf524 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:35.009264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7fa7d-1841-43b0-8cf0-44754f474ad9 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:35.022004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc73d74-b249-4da8-8612-b09077110448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:35.024720 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc73d74-b249-4da8-8612-b09077110448 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:35.025034 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc73d74-b249-4da8-8612-b09077110448 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:35.025295 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc73d74-b249-4da8-8612-b09077110448 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:35.028708 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc73d74-b249-4da8-8612-b09077110448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:35.032372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc73d74-b249-4da8-8612-b09077110448 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:35.067540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 845/1695] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:35.069410 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc73d74-b249-4da8-8612-b09077110448 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:35.086123 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 851/1696] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:35.091239 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfa701-74b4-4879-92a8-7d5ef3cdf524 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:35.098801 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc73d74-b249-4da8-8612-b09077110448 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:35.110916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e20af4-505d-4012-bfd6-df4c9467a374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:35.112732 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e20af4-505d-4012-bfd6-df4c9467a374 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:35.113054 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e20af4-505d-4012-bfd6-df4c9467a374 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:35.113854 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e20af4-505d-4012-bfd6-df4c9467a374 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:35.117352 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e20af4-505d-4012-bfd6-df4c9467a374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:35.119169 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e20af4-505d-4012-bfd6-df4c9467a374 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:35.137678 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfa701-74b4-4879-92a8-7d5ef3cdf524 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:35.149388 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 846/1697] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:35.156312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e20af4-505d-4012-bfd6-df4c9467a374 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:35.171988 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21beefac-b47b-4255-b199-ccd95d88cd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:35.173011 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21beefac-b47b-4255-b199-ccd95d88cd8f admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:35.179291 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21beefac-b47b-4255-b199-ccd95d88cd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:35.180839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21beefac-b47b-4255-b199-ccd95d88cd8f admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:35.183272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e20af4-505d-4012-bfd6-df4c9467a374 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:35.186809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21beefac-b47b-4255-b199-ccd95d88cd8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:35.193914 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21beefac-b47b-4255-b199-ccd95d88cd8f admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:35.202436 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12722a1-5f61-46ac-9cf4-363eb18bca58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:35.204361 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12722a1-5f61-46ac-9cf4-363eb18bca58 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:35.204697 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12722a1-5f61-46ac-9cf4-363eb18bca58 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:35.204945 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12722a1-5f61-46ac-9cf4-363eb18bca58 admin admin] PATH_INFO: `/v3/projects/80586e6c15ea4983b2606837706fa71f/users/53b97bb92bff496688e900b4e22bb258/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:35.273098 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12722a1-5f61-46ac-9cf4-363eb18bca58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80586e6c15ea4983b2606837706fa71f, user_id=53b97bb92bff496688e900b4e22bb258, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:35.277709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12722a1-5f61-46ac-9cf4-363eb18bca58 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:35.289857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 852/1698] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:35 2022] POST /identity/v3/projects => generated 365 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:35.300116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21beefac-b47b-4255-b199-ccd95d88cd8f admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:35.314753 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 847/1699] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:35 2022] PUT /identity/v3/projects/80586e6c15ea4983b2606837706fa71f/users/53b97bb92bff496688e900b4e22bb258/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:35.320640 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365bb7f-4982-41e3-8674-d0e1bcf4ab78 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:35.321295 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365bb7f-4982-41e3-8674-d0e1bcf4ab78 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:35.321622 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365bb7f-4982-41e3-8674-d0e1bcf4ab78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:35.359897 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-8365bb7f-4982-41e3-8674-d0e1bcf4ab78 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:35.361217 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-8365bb7f-4982-41e3-8674-d0e1bcf4ab78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:35.397047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:35.517750 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21beefac-b47b-4255-b199-ccd95d88cd8f admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:35.546163 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:35.593288 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 848/1700] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:35.653596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365bb7f-4982-41e3-8674-d0e1bcf4ab78 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:35.655405 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-8365bb7f-4982-41e3-8674-d0e1bcf4ab78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:35.689778 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f5dd8-ece7-4981-992a-c39537770de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:35.690497 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f5dd8-ece7-4981-992a-c39537770de9 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:35.690784 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f5dd8-ece7-4981-992a-c39537770de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:35.691094 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f5dd8-ece7-4981-992a-c39537770de9 admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:35.691802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f5dd8-ece7-4981-992a-c39537770de9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:35.709433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f5dd8-ece7-4981-992a-c39537770de9 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:35.779151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 853/1701] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:35 2022] POST /identity/v3/users => generated 283 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:35.782525 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f5dd8-ece7-4981-992a-c39537770de9 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:35.816691 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f5dd8-ece7-4981-992a-c39537770de9 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:35.855135 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428af4ff-f965-49af-a808-ce7bb51d452a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:35.858862 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428af4ff-f965-49af-a808-ce7bb51d452a admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:35.860617 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428af4ff-f965-49af-a808-ce7bb51d452a admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:35.861984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365bb7f-4982-41e3-8674-d0e1bcf4ab78 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:35.863046 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428af4ff-f965-49af-a808-ce7bb51d452a admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:35.864822 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428af4ff-f965-49af-a808-ce7bb51d452a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:35.870433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428af4ff-f965-49af-a808-ce7bb51d452a admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:35.883064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34819867-feb3-4c10-95ee-efd93998b7bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:35.900386 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34819867-feb3-4c10-95ee-efd93998b7bc service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:35.902828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34819867-feb3-4c10-95ee-efd93998b7bc service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:35.904524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34819867-feb3-4c10-95ee-efd93998b7bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:35.947383 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 854/1702] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:39:35.949143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428af4ff-f965-49af-a808-ce7bb51d452a admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:35.950366 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34819867-feb3-4c10-95ee-efd93998b7bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:35.953170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34819867-feb3-4c10-95ee-efd93998b7bc service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:35.994681 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 849/1703] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:36.003402 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428af4ff-f965-49af-a808-ce7bb51d452a admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:36.023199 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e04b0-97c9-49a6-9f0d-027a60652667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:36.027464 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e04b0-97c9-49a6-9f0d-027a60652667 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:36.029653 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e04b0-97c9-49a6-9f0d-027a60652667 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:36.029946 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e04b0-97c9-49a6-9f0d-027a60652667 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:36.032241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e04b0-97c9-49a6-9f0d-027a60652667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:36.039859 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e04b0-97c9-49a6-9f0d-027a60652667 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:36.134471 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 855/1704] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:36.139592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34819867-feb3-4c10-95ee-efd93998b7bc service neutron] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:36.166694 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34819867-feb3-4c10-95ee-efd93998b7bc service neutron] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:36.190649 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c27b37-bd05-4a42-ae3e-c4c60c7a3b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:36.194419 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c27b37-bd05-4a42-ae3e-c4c60c7a3b36 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:36.194810 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c27b37-bd05-4a42-ae3e-c4c60c7a3b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:36.195275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c27b37-bd05-4a42-ae3e-c4c60c7a3b36 admin admin] PATH_INFO: `/v3/projects/5ec5bb79b4b44f0c8ff0c684ab5c3dfd/users/353b8c9fd7c84e21b3f36d36a0590be0/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:36.238562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c27b37-bd05-4a42-ae3e-c4c60c7a3b36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ec5bb79b4b44f0c8ff0c684ab5c3dfd, user_id=353b8c9fd7c84e21b3f36d36a0590be0, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:36.250287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c27b37-bd05-4a42-ae3e-c4c60c7a3b36 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:36.308562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 850/1705] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:36 2022] PUT /identity/v3/projects/5ec5bb79b4b44f0c8ff0c684ab5c3dfd/users/353b8c9fd7c84e21b3f36d36a0590be0/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:36.318869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a21f27-5808-40a5-ac30-fa1a1aa15499 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:36.319187 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a21f27-5808-40a5-ac30-fa1a1aa15499 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:36.320431 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a21f27-5808-40a5-ac30-fa1a1aa15499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:36.425739 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-d0a21f27-5808-40a5-ac30-fa1a1aa15499 None None] MFA Rules not processed for user `353b8c9fd7c84e21b3f36d36a0590be0`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:36.427005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-d0a21f27-5808-40a5-ac30-fa1a1aa15499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:36.782641 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 856/1706] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:36.789421 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c27b37-bd05-4a42-ae3e-c4c60c7a3b36 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:37.042957 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c27b37-bd05-4a42-ae3e-c4c60c7a3b36 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:37.068435 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aff33a-a9a1-4d90-9a57-205e6d4f6a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:37.072011 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aff33a-a9a1-4d90-9a57-205e6d4f6a58 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:37.074066 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aff33a-a9a1-4d90-9a57-205e6d4f6a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:37.074423 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aff33a-a9a1-4d90-9a57-205e6d4f6a58 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:37.075953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aff33a-a9a1-4d90-9a57-205e6d4f6a58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:37.078040 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aff33a-a9a1-4d90-9a57-205e6d4f6a58 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:37.183370 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 851/1707] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:36 2022] POST /identity/v3/projects => generated 365 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:37.189007 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a21f27-5808-40a5-ac30-fa1a1aa15499 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:37.242921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a21f27-5808-40a5-ac30-fa1a1aa15499 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:37.420521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:37.497452 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc1568-8998-4f16-b806-d30aba09aa29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:37.498635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc1568-8998-4f16-b806-d30aba09aa29 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:37.500633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc1568-8998-4f16-b806-d30aba09aa29 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:37.502639 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc1568-8998-4f16-b806-d30aba09aa29 admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:37.503784 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc1568-8998-4f16-b806-d30aba09aa29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:37.510670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc1568-8998-4f16-b806-d30aba09aa29 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:37.570459 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 857/1708] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:37 2022] POST /identity/v3/users => generated 289 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:37.574759 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aff33a-a9a1-4d90-9a57-205e6d4f6a58 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:37.598884 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aff33a-a9a1-4d90-9a57-205e6d4f6a58 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:37.628355 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c9397-54c7-47b8-9a7e-a19cb88ba687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:37.630175 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c9397-54c7-47b8-9a7e-a19cb88ba687 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:37.630175 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c9397-54c7-47b8-9a7e-a19cb88ba687 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:37.630478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c9397-54c7-47b8-9a7e-a19cb88ba687 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:37.634466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3c9397-54c7-47b8-9a7e-a19cb88ba687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:37.637353 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3c9397-54c7-47b8-9a7e-a19cb88ba687 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:37.691915 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc1568-8998-4f16-b806-d30aba09aa29 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:37.700595 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 852/1709] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:37.707790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc1568-8998-4f16-b806-d30aba09aa29 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:37.728913 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378c0cc-ebb7-4d63-8ff3-31c038b20aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:37.729921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378c0cc-ebb7-4d63-8ff3-31c038b20aae admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:37.730203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378c0cc-ebb7-4d63-8ff3-31c038b20aae admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:37.730478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378c0cc-ebb7-4d63-8ff3-31c038b20aae admin admin] PATH_INFO: `/v3/projects/6563c3c96b674a50a567e07bbe548c5a/users/d5d3d14053ec49ca992ea2b3166cc699/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:37.783116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378c0cc-ebb7-4d63-8ff3-31c038b20aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=6563c3c96b674a50a567e07bbe548c5a, user_id=d5d3d14053ec49ca992ea2b3166cc699, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:37.785945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378c0cc-ebb7-4d63-8ff3-31c038b20aae admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:37.826011 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 858/1710] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:37 2022] PUT /identity/v3/projects/6563c3c96b674a50a567e07bbe548c5a/users/d5d3d14053ec49ca992ea2b3166cc699/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:39:37.831340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c9397-54c7-47b8-9a7e-a19cb88ba687 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:37.850777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c9397-54c7-47b8-9a7e-a19cb88ba687 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:38.033045 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:38.152846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae9060-44d3-47f5-bd2c-2589d024bb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:38.159268 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae9060-44d3-47f5-bd2c-2589d024bb26 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:38.162301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae9060-44d3-47f5-bd2c-2589d024bb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:38.162952 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae9060-44d3-47f5-bd2c-2589d024bb26 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:38.167074 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae9060-44d3-47f5-bd2c-2589d024bb26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:38.174406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae9060-44d3-47f5-bd2c-2589d024bb26 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:38.245630 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 853/1711] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:38.256444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378c0cc-ebb7-4d63-8ff3-31c038b20aae admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:38.275407 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378c0cc-ebb7-4d63-8ff3-31c038b20aae admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:38.311040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48dd12-b1b5-4580-837e-2b8b3102b86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:38.314208 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48dd12-b1b5-4580-837e-2b8b3102b86e admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:38.314840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48dd12-b1b5-4580-837e-2b8b3102b86e admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:38.315901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48dd12-b1b5-4580-837e-2b8b3102b86e admin admin] PATH_INFO: `/v3/projects/6563c3c96b674a50a567e07bbe548c5a/users/d5d3d14053ec49ca992ea2b3166cc699/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:38.321590 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48dd12-b1b5-4580-837e-2b8b3102b86e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6563c3c96b674a50a567e07bbe548c5a, user_id=d5d3d14053ec49ca992ea2b3166cc699, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:38.331689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48dd12-b1b5-4580-837e-2b8b3102b86e admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:38.386371 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 859/1712] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:38 2022] PUT /identity/v3/projects/6563c3c96b674a50a567e07bbe548c5a/users/d5d3d14053ec49ca992ea2b3166cc699/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:38.402423 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440af66-0307-4d1e-87af-fe4c6a6dccdb None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:38.403042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440af66-0307-4d1e-87af-fe4c6a6dccdb None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:38.403511 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440af66-0307-4d1e-87af-fe4c6a6dccdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:38.467035 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-7440af66-0307-4d1e-87af-fe4c6a6dccdb None None] MFA Rules not processed for user `d5d3d14053ec49ca992ea2b3166cc699`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:38.468572 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-7440af66-0307-4d1e-87af-fe4c6a6dccdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:38.707117 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 854/1713] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:38 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:39:38.728048 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48dd12-b1b5-4580-837e-2b8b3102b86e admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:38.730174 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-1a48dd12-b1b5-4580-837e-2b8b3102b86e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:38.904119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48dd12-b1b5-4580-837e-2b8b3102b86e admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:38.913560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96baab92-ac15-4396-b6b9-643e6d4eaef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:38.915357 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96baab92-ac15-4396-b6b9-643e6d4eaef7 service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:38.916106 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96baab92-ac15-4396-b6b9-643e6d4eaef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:38.918462 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96baab92-ac15-4396-b6b9-643e6d4eaef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:38.940577 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96baab92-ac15-4396-b6b9-643e6d4eaef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:38.944110 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96baab92-ac15-4396-b6b9-643e6d4eaef7 service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:38.984782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 860/1714] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:38 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:39.881820 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b1820-2201-463f-85d9-b5aeab19608c None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:39.882263 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b1820-2201-463f-85d9-b5aeab19608c None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:39.882377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b1820-2201-463f-85d9-b5aeab19608c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:39.926458 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-aa7b1820-2201-463f-85d9-b5aeab19608c None None] MFA Rules not processed for user `03eee8ef92314296b7d5c14c52f0d080`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:39.930115 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-aa7b1820-2201-463f-85d9-b5aeab19608c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:40.303197 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 855/1715] 10.222.0.79 () {58 vars in 1015 bytes} [Thu Jul 21 20:39:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:40.555059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96baab92-ac15-4396-b6b9-643e6d4eaef7 service cinder] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:40.572686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96baab92-ac15-4396-b6b9-643e6d4eaef7 service cinder] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:40.596940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b08cd7-047f-4612-8c5e-2fa634a417e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:40.602347 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b08cd7-047f-4612-8c5e-2fa634a417e6 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:40.602647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b08cd7-047f-4612-8c5e-2fa634a417e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:40.602898 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b08cd7-047f-4612-8c5e-2fa634a417e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:40.650431 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b08cd7-047f-4612-8c5e-2fa634a417e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:40.652162 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b08cd7-047f-4612-8c5e-2fa634a417e6 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:40.694817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 861/1716] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:41.631953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b1820-2201-463f-85d9-b5aeab19608c None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:41.822703 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b1820-2201-463f-85d9-b5aeab19608c None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:41.837365 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f40e3-0e4d-4c0d-81ce-fbadf9d35a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:41.838973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f40e3-0e4d-4c0d-81ce-fbadf9d35a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:41.839599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f40e3-0e4d-4c0d-81ce-fbadf9d35a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:41.840880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f40e3-0e4d-4c0d-81ce-fbadf9d35a48 admin admin] PATH_INFO: `/v3/users/353b8c9fd7c84e21b3f36d36a0590be0` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:41.845661 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f40e3-0e4d-4c0d-81ce-fbadf9d35a48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=353b8c9fd7c84e21b3f36d36a0590be0)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:41.852323 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f40e3-0e4d-4c0d-81ce-fbadf9d35a48 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:41.891782 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:42.024156 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-fc9f40e3-0e4d-4c0d-81ce-fbadf9d35a48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '353b8c9fd7c84e21b3f36d36a0590be0'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:42.037664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-fc9f40e3-0e4d-4c0d-81ce-fbadf9d35a48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '353b8c9fd7c84e21b3f36d36a0590be0'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:42.037843 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-fc9f40e3-0e4d-4c0d-81ce-fbadf9d35a48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '353b8c9fd7c84e21b3f36d36a0590be0'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:42.102274 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-fc9f40e3-0e4d-4c0d-81ce-fbadf9d35a48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '353b8c9fd7c84e21b3f36d36a0590be0'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:42.133466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-fc9f40e3-0e4d-4c0d-81ce-fbadf9d35a48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '353b8c9fd7c84e21b3f36d36a0590be0'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:42.144491 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 856/1717] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:39:41 2022] DELETE /identity/v3/users/353b8c9fd7c84e21b3f36d36a0590be0 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:42.149635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39acee-763b-499c-ab92-e9a7548e3d6e None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:42.150561 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39acee-763b-499c-ab92-e9a7548e3d6e None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:42.150910 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39acee-763b-499c-ab92-e9a7548e3d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:42.201464 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-cd39acee-763b-499c-ab92-e9a7548e3d6e None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:42.206297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-cd39acee-763b-499c-ab92-e9a7548e3d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:42.436410 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 862/1718] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:42.535781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f40e3-0e4d-4c0d-81ce-fbadf9d35a48 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:42.537304 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-fc9f40e3-0e4d-4c0d-81ce-fbadf9d35a48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:42.800038 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f40e3-0e4d-4c0d-81ce-fbadf9d35a48 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:42.817478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ef302-22c7-4a84-8dec-657822d9ad2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:42.818571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ef302-22c7-4a84-8dec-657822d9ad2b service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:42.819325 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ef302-22c7-4a84-8dec-657822d9ad2b service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:42.819872 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ef302-22c7-4a84-8dec-657822d9ad2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:42.856588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ef302-22c7-4a84-8dec-657822d9ad2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:42.858392 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ef302-22c7-4a84-8dec-657822d9ad2b service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:42.871166 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 857/1719] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:43.587404 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39acee-763b-499c-ab92-e9a7548e3d6e None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:43.589286 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-cd39acee-763b-499c-ab92-e9a7548e3d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:43.624810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39acee-763b-499c-ab92-e9a7548e3d6e None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:43.663362 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac51c32-3b28-4b01-97ed-a062784985b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:43.668843 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac51c32-3b28-4b01-97ed-a062784985b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:43.670913 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac51c32-3b28-4b01-97ed-a062784985b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:43.674058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac51c32-3b28-4b01-97ed-a062784985b6 admin admin] PATH_INFO: `/v3/users/ad0d36d8ab824590a3c0c9ff92b2d7a9` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:43.678954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac51c32-3b28-4b01-97ed-a062784985b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad0d36d8ab824590a3c0c9ff92b2d7a9)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:43.701120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac51c32-3b28-4b01-97ed-a062784985b6 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:43.763594 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:43.946678 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-bac51c32-3b28-4b01-97ed-a062784985b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad0d36d8ab824590a3c0c9ff92b2d7a9'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:43.973295 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-bac51c32-3b28-4b01-97ed-a062784985b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad0d36d8ab824590a3c0c9ff92b2d7a9'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:43.974989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-bac51c32-3b28-4b01-97ed-a062784985b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad0d36d8ab824590a3c0c9ff92b2d7a9'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:44.003806 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-bac51c32-3b28-4b01-97ed-a062784985b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad0d36d8ab824590a3c0c9ff92b2d7a9'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:44.006125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-bac51c32-3b28-4b01-97ed-a062784985b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad0d36d8ab824590a3c0c9ff92b2d7a9'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:44.077237 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 863/1720] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:39:43 2022] DELETE /identity/v3/users/ad0d36d8ab824590a3c0c9ff92b2d7a9 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:44.083378 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8b97a-bd38-4d9e-bfd4-2a726ceeae58 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:44.083960 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8b97a-bd38-4d9e-bfd4-2a726ceeae58 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:44.083960 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8b97a-bd38-4d9e-bfd4-2a726ceeae58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:44.127203 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-15c8b97a-bd38-4d9e-bfd4-2a726ceeae58 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:44.147744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-15c8b97a-bd38-4d9e-bfd4-2a726ceeae58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:44.305172 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 858/1721] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:44.346259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac51c32-3b28-4b01-97ed-a062784985b6 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:44.348188 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-bac51c32-3b28-4b01-97ed-a062784985b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:44.511488 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b2e32-8fdb-4153-a66d-25c1b649ac43 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:44.511863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b2e32-8fdb-4153-a66d-25c1b649ac43 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:44.512088 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b2e32-8fdb-4153-a66d-25c1b649ac43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:44.665501 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-592b2e32-8fdb-4153-a66d-25c1b649ac43 None None] MFA Rules not processed for user `53b97bb92bff496688e900b4e22bb258`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:44.679850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-592b2e32-8fdb-4153-a66d-25c1b649ac43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:44.685256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac51c32-3b28-4b01-97ed-a062784985b6 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:44.694091 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300deeaa-079d-4cb0-b917-0a4ee8209970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:44.694976 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300deeaa-079d-4cb0-b917-0a4ee8209970 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:44.695755 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300deeaa-079d-4cb0-b917-0a4ee8209970 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:44.696189 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300deeaa-079d-4cb0-b917-0a4ee8209970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:44.722317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300deeaa-079d-4cb0-b917-0a4ee8209970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:44.723703 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300deeaa-079d-4cb0-b917-0a4ee8209970 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:44.750257 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 864/1722] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:44.810722 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:44.947014 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 859/1723] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:44.951560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300deeaa-079d-4cb0-b917-0a4ee8209970 service neutron] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:44.953777 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-300deeaa-079d-4cb0-b917-0a4ee8209970 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:44.986406 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300deeaa-079d-4cb0-b917-0a4ee8209970 service neutron] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:44.999318 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94d787-80fe-469c-b35c-847b18ea9380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:45.000407 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94d787-80fe-469c-b35c-847b18ea9380 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:45.000977 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94d787-80fe-469c-b35c-847b18ea9380 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:45.002476 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94d787-80fe-469c-b35c-847b18ea9380 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:45.003684 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94d787-80fe-469c-b35c-847b18ea9380 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:45.010416 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94d787-80fe-469c-b35c-847b18ea9380 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:45.079731 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 865/1724] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:44 2022] POST /identity/v3/projects => generated 403 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:45.084104 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b2e32-8fdb-4153-a66d-25c1b649ac43 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:45.102773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b2e32-8fdb-4153-a66d-25c1b649ac43 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:45.311776 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7285bc-d92f-4fb3-b9cf-c13e912631a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:45.312754 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7285bc-d92f-4fb3-b9cf-c13e912631a2 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:45.313236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7285bc-d92f-4fb3-b9cf-c13e912631a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:45.313674 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7285bc-d92f-4fb3-b9cf-c13e912631a2 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:45.314570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7285bc-d92f-4fb3-b9cf-c13e912631a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:45.322322 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7285bc-d92f-4fb3-b9cf-c13e912631a2 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:45.379002 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 860/1725] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:45 2022] POST /identity/v3/users => generated 308 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:45.380710 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94d787-80fe-469c-b35c-847b18ea9380 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:45.405933 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94d787-80fe-469c-b35c-847b18ea9380 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:45.412787 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58d3ea-f234-4066-bbde-413adc9f5df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:45.413524 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58d3ea-f234-4066-bbde-413adc9f5df1 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:45.413809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58d3ea-f234-4066-bbde-413adc9f5df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:45.414086 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58d3ea-f234-4066-bbde-413adc9f5df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:45.414832 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe58d3ea-f234-4066-bbde-413adc9f5df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:45.416479 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe58d3ea-f234-4066-bbde-413adc9f5df1 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:45.448043 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 866/1726] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:45.452780 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7285bc-d92f-4fb3-b9cf-c13e912631a2 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:45.454012 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58d3ea-f234-4066-bbde-413adc9f5df1 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:45.462015 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-fe58d3ea-f234-4066-bbde-413adc9f5df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:45.479349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7285bc-d92f-4fb3-b9cf-c13e912631a2 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:45.516170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc4428-b1b8-44fa-8bce-6ec79aa9b531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:45.517111 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58d3ea-f234-4066-bbde-413adc9f5df1 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:45.517245 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc4428-b1b8-44fa-8bce-6ec79aa9b531 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:45.517515 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc4428-b1b8-44fa-8bce-6ec79aa9b531 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:45.517792 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc4428-b1b8-44fa-8bce-6ec79aa9b531 admin admin] PATH_INFO: `/v3/projects/0cae91ea906b4b09a6bcc3cb3545a078/users/19551e1cc3ce406dbb565de2516c3b9f/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:45.533896 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334abab-55a7-4449-96d9-d25505de9e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:45.534795 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334abab-55a7-4449-96d9-d25505de9e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:45.535081 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334abab-55a7-4449-96d9-d25505de9e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:45.535366 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334abab-55a7-4449-96d9-d25505de9e0e admin admin] PATH_INFO: `/v3/projects/5ec5bb79b4b44f0c8ff0c684ab5c3dfd` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:45.537655 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334abab-55a7-4449-96d9-d25505de9e0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ec5bb79b4b44f0c8ff0c684ab5c3dfd)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:45.539633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334abab-55a7-4449-96d9-d25505de9e0e admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:45.559032 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fc4428-b1b8-44fa-8bce-6ec79aa9b531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cae91ea906b4b09a6bcc3cb3545a078, user_id=19551e1cc3ce406dbb565de2516c3b9f, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:45.563711 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fc4428-b1b8-44fa-8bce-6ec79aa9b531 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:45.655444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 861/1727] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:45 2022] PUT /identity/v3/projects/0cae91ea906b4b09a6bcc3cb3545a078/users/19551e1cc3ce406dbb565de2516c3b9f/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:45.657257 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99728d1f-23e4-43be-a3b2-d5ff52a9cb05 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:45.657571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99728d1f-23e4-43be-a3b2-d5ff52a9cb05 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:45.657853 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99728d1f-23e4-43be-a3b2-d5ff52a9cb05 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:45.664899 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2334abab-55a7-4449-96d9-d25505de9e0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ec5bb79b4b44f0c8ff0c684ab5c3dfd', 'request_id': 'req-2334abab-55a7-4449-96d9-d25505de9e0e'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:45.672406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 862/1728] 10.222.0.71 () {54 vars in 921 bytes} [Thu Jul 21 20:39:45 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:45.677349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99728d1f-23e4-43be-a3b2-d5ff52a9cb05 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:45.679948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:45.704438 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2334abab-55a7-4449-96d9-d25505de9e0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ec5bb79b4b44f0c8ff0c684ab5c3dfd', 'request_id': 'req-2334abab-55a7-4449-96d9-d25505de9e0e'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:45.783198 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2334abab-55a7-4449-96d9-d25505de9e0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ec5bb79b4b44f0c8ff0c684ab5c3dfd', 'request_id': 'req-2334abab-55a7-4449-96d9-d25505de9e0e'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:45.898951 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2334abab-55a7-4449-96d9-d25505de9e0e admin admin] The token cache is being invalidate because project 5ec5bb79b4b44f0c8ff0c684ab5c3dfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:39:45.899308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2334abab-55a7-4449-96d9-d25505de9e0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:45.904477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2334abab-55a7-4449-96d9-d25505de9e0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:45.919578 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 867/1729] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:39:45 2022] DELETE /identity/v3/projects/5ec5bb79b4b44f0c8ff0c684ab5c3dfd => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:45.928224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361850c-4b8a-4be8-8367-8505b915e678 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:45.928713 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361850c-4b8a-4be8-8367-8505b915e678 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:45.929032 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361850c-4b8a-4be8-8367-8505b915e678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:46.039487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-3361850c-4b8a-4be8-8367-8505b915e678 None None] MFA Rules not processed for user `6fbc5240f5214aaea4418052b94739e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:46.046117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3361850c-4b8a-4be8-8367-8505b915e678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:46.135951 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99728d1f-23e4-43be-a3b2-d5ff52a9cb05 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:46.151932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609ca98-d740-44fe-9441-5c8db131b53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:46.152810 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609ca98-d740-44fe-9441-5c8db131b53b admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:46.154638 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609ca98-d740-44fe-9441-5c8db131b53b admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:46.155675 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609ca98-d740-44fe-9441-5c8db131b53b admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:46.156561 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609ca98-d740-44fe-9441-5c8db131b53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:46.159477 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609ca98-d740-44fe-9441-5c8db131b53b admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:46.187864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 863/1730] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 515 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:46.190768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609ca98-d740-44fe-9441-5c8db131b53b admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:46.214130 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c609ca98-d740-44fe-9441-5c8db131b53b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:46.242016 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609ca98-d740-44fe-9441-5c8db131b53b admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:46.250518 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 868/1731] 10.222.0.71 () {58 vars in 1018 bytes} [Thu Jul 21 20:39:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:46.251422 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361850c-4b8a-4be8-8367-8505b915e678 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:46.252851 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3361850c-4b8a-4be8-8367-8505b915e678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:46.253301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02c8fc-b56f-4d22-9dbc-bda9ac26e6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:46.254199 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02c8fc-b56f-4d22-9dbc-bda9ac26e6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:46.254700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02c8fc-b56f-4d22-9dbc-bda9ac26e6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:46.255176 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02c8fc-b56f-4d22-9dbc-bda9ac26e6fd admin admin] PATH_INFO: `/v3/users/d5d3d14053ec49ca992ea2b3166cc699` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:46.259234 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02c8fc-b56f-4d22-9dbc-bda9ac26e6fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5d3d14053ec49ca992ea2b3166cc699)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:46.261821 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02c8fc-b56f-4d22-9dbc-bda9ac26e6fd admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:46.297671 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361850c-4b8a-4be8-8367-8505b915e678 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:46.309269 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9d797-be45-4dea-9933-c73ab10289d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:46.310422 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9d797-be45-4dea-9933-c73ab10289d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:46.310883 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9d797-be45-4dea-9933-c73ab10289d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:46.311320 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9d797-be45-4dea-9933-c73ab10289d8 admin admin] PATH_INFO: `/v3/projects/0cae91ea906b4b09a6bcc3cb3545a078/users/19551e1cc3ce406dbb565de2516c3b9f/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:46.317783 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9d797-be45-4dea-9933-c73ab10289d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cae91ea906b4b09a6bcc3cb3545a078, user_id=19551e1cc3ce406dbb565de2516c3b9f, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:46.320752 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:46.321007 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9d797-be45-4dea-9933-c73ab10289d8 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:46.374058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 869/1732] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:46 2022] PUT /identity/v3/projects/0cae91ea906b4b09a6bcc3cb3545a078/users/19551e1cc3ce406dbb565de2516c3b9f/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:46.375436 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9d797-be45-4dea-9933-c73ab10289d8 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:46.380141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-e8b9d797-be45-4dea-9933-c73ab10289d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:46.390546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-6d02c8fc-b56f-4d22-9dbc-bda9ac26e6fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5d3d14053ec49ca992ea2b3166cc699'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:46.410668 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-6d02c8fc-b56f-4d22-9dbc-bda9ac26e6fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5d3d14053ec49ca992ea2b3166cc699'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:46.411462 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-6d02c8fc-b56f-4d22-9dbc-bda9ac26e6fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5d3d14053ec49ca992ea2b3166cc699'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:46.466088 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-6d02c8fc-b56f-4d22-9dbc-bda9ac26e6fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5d3d14053ec49ca992ea2b3166cc699'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:46.510157 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-6d02c8fc-b56f-4d22-9dbc-bda9ac26e6fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5d3d14053ec49ca992ea2b3166cc699'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:46.540800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 864/1733] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:39:46 2022] DELETE /identity/v3/users/d5d3d14053ec49ca992ea2b3166cc699 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:46.605202 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9d797-be45-4dea-9933-c73ab10289d8 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:46.722365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:46.753537 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09800f-5fc1-47de-a331-f8392d727a12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:46.754219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09800f-5fc1-47de-a331-f8392d727a12 service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:46.754505 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09800f-5fc1-47de-a331-f8392d727a12 service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:46.754780 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09800f-5fc1-47de-a331-f8392d727a12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:46.768854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-fc09800f-5fc1-47de-a331-f8392d727a12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:46.801222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09800f-5fc1-47de-a331-f8392d727a12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:46.803316 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09800f-5fc1-47de-a331-f8392d727a12 service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:46.821413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 870/1734] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:47.546797 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02c8fc-b56f-4d22-9dbc-bda9ac26e6fd admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:47.548713 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-6d02c8fc-b56f-4d22-9dbc-bda9ac26e6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:47.776882 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02c8fc-b56f-4d22-9dbc-bda9ac26e6fd admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:47.805932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8b572-fdaa-48d0-a1a6-5c0205172bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:47.807367 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8b572-fdaa-48d0-a1a6-5c0205172bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:47.808427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8b572-fdaa-48d0-a1a6-5c0205172bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:47.808848 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8b572-fdaa-48d0-a1a6-5c0205172bbd admin admin] PATH_INFO: `/v3/projects/461d19c7d1a4450eb4a9fd923dd13f7c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:47.811102 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8b572-fdaa-48d0-a1a6-5c0205172bbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=461d19c7d1a4450eb4a9fd923dd13f7c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:47.813418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8b572-fdaa-48d0-a1a6-5c0205172bbd admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:47.963870 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-bde8b572-fdaa-48d0-a1a6-5c0205172bbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '461d19c7d1a4450eb4a9fd923dd13f7c', 'request_id': 'req-bde8b572-fdaa-48d0-a1a6-5c0205172bbd'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:47.983992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-bde8b572-fdaa-48d0-a1a6-5c0205172bbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '461d19c7d1a4450eb4a9fd923dd13f7c', 'request_id': 'req-bde8b572-fdaa-48d0-a1a6-5c0205172bbd'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:48.008534 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-bde8b572-fdaa-48d0-a1a6-5c0205172bbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '461d19c7d1a4450eb4a9fd923dd13f7c', 'request_id': 'req-bde8b572-fdaa-48d0-a1a6-5c0205172bbd'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:48.049137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-bde8b572-fdaa-48d0-a1a6-5c0205172bbd admin admin] The token cache is being invalidate because project 461d19c7d1a4450eb4a9fd923dd13f7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:39:48.050360 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-bde8b572-fdaa-48d0-a1a6-5c0205172bbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:48.052702 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-bde8b572-fdaa-48d0-a1a6-5c0205172bbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:48.058982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 865/1735] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:39:47 2022] DELETE /identity/v3/projects/461d19c7d1a4450eb4a9fd923dd13f7c => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:48.063205 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09800f-5fc1-47de-a331-f8392d727a12 service nova] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:48.090228 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09800f-5fc1-47de-a331-f8392d727a12 service nova] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:48.315091 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f52ef-cdd4-40a6-aecc-a91bb76fc8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:48.319415 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f52ef-cdd4-40a6-aecc-a91bb76fc8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:48.321155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f52ef-cdd4-40a6-aecc-a91bb76fc8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:48.321733 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f52ef-cdd4-40a6-aecc-a91bb76fc8d2 admin admin] PATH_INFO: `/v3/users/e6ac301f616f437b9f790757e2390fac` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:48.328101 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f52ef-cdd4-40a6-aecc-a91bb76fc8d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6ac301f616f437b9f790757e2390fac)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:48.338197 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f52ef-cdd4-40a6-aecc-a91bb76fc8d2 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:48.601063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-ba5f52ef-cdd4-40a6-aecc-a91bb76fc8d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6ac301f616f437b9f790757e2390fac'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:48.661729 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-ba5f52ef-cdd4-40a6-aecc-a91bb76fc8d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6ac301f616f437b9f790757e2390fac'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:48.667687 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-ba5f52ef-cdd4-40a6-aecc-a91bb76fc8d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6ac301f616f437b9f790757e2390fac'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:48.725897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-ba5f52ef-cdd4-40a6-aecc-a91bb76fc8d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6ac301f616f437b9f790757e2390fac'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:48.728821 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-ba5f52ef-cdd4-40a6-aecc-a91bb76fc8d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6ac301f616f437b9f790757e2390fac'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:48.747527 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f94f0b-5861-4ec0-b01d-9602cefcb241 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:48.753004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f94f0b-5861-4ec0-b01d-9602cefcb241 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:48.755315 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f94f0b-5861-4ec0-b01d-9602cefcb241 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:48.762652 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 866/1736] 10.222.0.71 () {54 vars in 921 bytes} [Thu Jul 21 20:39:48 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:48.809700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 871/1737] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:39:48 2022] DELETE /identity/v3/users/e6ac301f616f437b9f790757e2390fac => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:48.988517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e8457-cd97-40c3-9ea6-f307b69f03d2 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:48.989481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e8457-cd97-40c3-9ea6-f307b69f03d2 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:48.990380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e8457-cd97-40c3-9ea6-f307b69f03d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:49.108795 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-508e8457-cd97-40c3-9ea6-f307b69f03d2 None None] MFA Rules not processed for user `6fbc5240f5214aaea4418052b94739e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:49.116088 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-508e8457-cd97-40c3-9ea6-f307b69f03d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:49.198466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:49.378744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 867/1738] 10.222.0.71 () {58 vars in 1018 bytes} [Thu Jul 21 20:39:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:39:50.417123 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f52ef-cdd4-40a6-aecc-a91bb76fc8d2 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:50.420291 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-ba5f52ef-cdd4-40a6-aecc-a91bb76fc8d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:50.703801 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f52ef-cdd4-40a6-aecc-a91bb76fc8d2 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:50.735805 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53582808-b6e1-4f35-9c5a-8ab3d326406e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:50.736746 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53582808-b6e1-4f35-9c5a-8ab3d326406e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:50.737056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53582808-b6e1-4f35-9c5a-8ab3d326406e admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:50.739258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53582808-b6e1-4f35-9c5a-8ab3d326406e admin admin] PATH_INFO: `/v3/projects/6563c3c96b674a50a567e07bbe548c5a` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:50.742152 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53582808-b6e1-4f35-9c5a-8ab3d326406e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6563c3c96b674a50a567e07bbe548c5a)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:50.748204 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53582808-b6e1-4f35-9c5a-8ab3d326406e admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:50.775085 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:50.909213 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-53582808-b6e1-4f35-9c5a-8ab3d326406e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6563c3c96b674a50a567e07bbe548c5a', 'request_id': 'req-53582808-b6e1-4f35-9c5a-8ab3d326406e'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:50.933022 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-53582808-b6e1-4f35-9c5a-8ab3d326406e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6563c3c96b674a50a567e07bbe548c5a', 'request_id': 'req-53582808-b6e1-4f35-9c5a-8ab3d326406e'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:50.965681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-53582808-b6e1-4f35-9c5a-8ab3d326406e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6563c3c96b674a50a567e07bbe548c5a', 'request_id': 'req-53582808-b6e1-4f35-9c5a-8ab3d326406e'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:51.042950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-53582808-b6e1-4f35-9c5a-8ab3d326406e admin admin] The token cache is being invalidate because project 6563c3c96b674a50a567e07bbe548c5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:39:51.046082 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-53582808-b6e1-4f35-9c5a-8ab3d326406e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:51.048047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-53582808-b6e1-4f35-9c5a-8ab3d326406e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:51.055877 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 872/1739] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:39:50 2022] DELETE /identity/v3/projects/6563c3c96b674a50a567e07bbe548c5a => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:51.062859 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de2524-2b82-4c92-811f-2aad2f34cd9f None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:51.063733 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de2524-2b82-4c92-811f-2aad2f34cd9f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:51.064130 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de2524-2b82-4c92-811f-2aad2f34cd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:51.177981 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-d0de2524-2b82-4c92-811f-2aad2f34cd9f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:51.198890 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d0de2524-2b82-4c92-811f-2aad2f34cd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:51.332113 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53582808-b6e1-4f35-9c5a-8ab3d326406e admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:51.334172 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-53582808-b6e1-4f35-9c5a-8ab3d326406e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:51.657490 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 868/1740] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 597 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:39:51.674090 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dc57d-2909-41d0-98d1-386185d244ce None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:51.674614 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dc57d-2909-41d0-98d1-386185d244ce None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:51.675185 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dc57d-2909-41d0-98d1-386185d244ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:51.742693 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-cf6dc57d-2909-41d0-98d1-386185d244ce None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:51.743967 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-cf6dc57d-2909-41d0-98d1-386185d244ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:51.766355 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 869/1741] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:39:51.766839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6dc57d-2909-41d0-98d1-386185d244ce None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:51.786833 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6dc57d-2909-41d0-98d1-386185d244ce None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:51.793769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be668640-8fe1-4ebe-ad4a-ecb500ec1f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:51.794273 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be668640-8fe1-4ebe-ad4a-ecb500ec1f4e admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:51.794563 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be668640-8fe1-4ebe-ad4a-ecb500ec1f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:51.794841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be668640-8fe1-4ebe-ad4a-ecb500ec1f4e admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:51.795810 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be668640-8fe1-4ebe-ad4a-ecb500ec1f4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:51.796304 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be668640-8fe1-4ebe-ad4a-ecb500ec1f4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:51.798313 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be668640-8fe1-4ebe-ad4a-ecb500ec1f4e admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:51.799808 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53582808-b6e1-4f35-9c5a-8ab3d326406e admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:51.818753 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf099c8-e9d4-4853-9443-9df1cccd13bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:51.819425 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf099c8-e9d4-4853-9443-9df1cccd13bf service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:51.819725 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf099c8-e9d4-4853-9443-9df1cccd13bf service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:51.820030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf099c8-e9d4-4853-9443-9df1cccd13bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:51.823310 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 870/1742] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:51.824573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be668640-8fe1-4ebe-ad4a-ecb500ec1f4e admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:51.831452 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-1bf099c8-e9d4-4853-9443-9df1cccd13bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:51.845877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be668640-8fe1-4ebe-ad4a-ecb500ec1f4e admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:51.853809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651da48-0f71-49ef-b0c8-39a31c645197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:51.855857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651da48-0f71-49ef-b0c8-39a31c645197 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:51.856866 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651da48-0f71-49ef-b0c8-39a31c645197 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:51.857404 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651da48-0f71-49ef-b0c8-39a31c645197 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:51.859466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651da48-0f71-49ef-b0c8-39a31c645197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:51.868741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651da48-0f71-49ef-b0c8-39a31c645197 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:51.871790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf099c8-e9d4-4853-9443-9df1cccd13bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:51.876382 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf099c8-e9d4-4853-9443-9df1cccd13bf service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:51.915979 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 871/1743] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:51 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:51.919335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 873/1744] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:51.923347 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651da48-0f71-49ef-b0c8-39a31c645197 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:52.030902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5281309-e25c-4aef-afc4-95f9b2ef67b9 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.031493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5281309-e25c-4aef-afc4-95f9b2ef67b9 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.031602 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5281309-e25c-4aef-afc4-95f9b2ef67b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.117465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-e5281309-e25c-4aef-afc4-95f9b2ef67b9 None None] MFA Rules not processed for user `19551e1cc3ce406dbb565de2516c3b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:52.122196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-e5281309-e25c-4aef-afc4-95f9b2ef67b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:52.193039 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651da48-0f71-49ef-b0c8-39a31c645197 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:52.203736 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc053cb0-bcd2-4e34-8956-bf72b2212d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:52.205279 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc053cb0-bcd2-4e34-8956-bf72b2212d77 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.205741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc053cb0-bcd2-4e34-8956-bf72b2212d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.210161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc053cb0-bcd2-4e34-8956-bf72b2212d77 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.211233 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc053cb0-bcd2-4e34-8956-bf72b2212d77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:52.213614 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc053cb0-bcd2-4e34-8956-bf72b2212d77 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:52.277291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 872/1745] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:51 2022] POST /identity/v3/users => generated 288 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:52.282949 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc053cb0-bcd2-4e34-8956-bf72b2212d77 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:52.288211 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-fc053cb0-bcd2-4e34-8956-bf72b2212d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:52.317187 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc053cb0-bcd2-4e34-8956-bf72b2212d77 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:52.329261 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e1726-2dab-4ec6-98ca-1326fb880df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:52.334636 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e1726-2dab-4ec6-98ca-1326fb880df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.335247 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e1726-2dab-4ec6-98ca-1326fb880df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.335738 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e1726-2dab-4ec6-98ca-1326fb880df6 admin admin] PATH_INFO: `/v3/projects/0658ab86ba2c4fb8a8b24ace9aaa66b0` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.337535 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e1726-2dab-4ec6-98ca-1326fb880df6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0658ab86ba2c4fb8a8b24ace9aaa66b0)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:52.345265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e1726-2dab-4ec6-98ca-1326fb880df6 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:52.352746 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:52.358384 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 874/1746] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:52 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:52.359245 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5281309-e25c-4aef-afc4-95f9b2ef67b9 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:52.379804 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:52.400033 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5281309-e25c-4aef-afc4-95f9b2ef67b9 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:52.406185 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07f8dd-f7bd-4c6b-8f01-ae9dd38b0877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:52.407279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07f8dd-f7bd-4c6b-8f01-ae9dd38b0877 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.408150 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07f8dd-f7bd-4c6b-8f01-ae9dd38b0877 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.408600 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07f8dd-f7bd-4c6b-8f01-ae9dd38b0877 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.409621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07f8dd-f7bd-4c6b-8f01-ae9dd38b0877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:52.413420 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07f8dd-f7bd-4c6b-8f01-ae9dd38b0877 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:52.435837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 875/1747] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:52.441903 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07f8dd-f7bd-4c6b-8f01-ae9dd38b0877 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:52.472024 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07f8dd-f7bd-4c6b-8f01-ae9dd38b0877 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:52.502406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-261e1726-2dab-4ec6-98ca-1326fb880df6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0658ab86ba2c4fb8a8b24ace9aaa66b0', 'request_id': 'req-261e1726-2dab-4ec6-98ca-1326fb880df6'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:52.526020 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-261e1726-2dab-4ec6-98ca-1326fb880df6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0658ab86ba2c4fb8a8b24ace9aaa66b0', 'request_id': 'req-261e1726-2dab-4ec6-98ca-1326fb880df6'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:52.538725 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-261e1726-2dab-4ec6-98ca-1326fb880df6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0658ab86ba2c4fb8a8b24ace9aaa66b0', 'request_id': 'req-261e1726-2dab-4ec6-98ca-1326fb880df6'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:52.568924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-261e1726-2dab-4ec6-98ca-1326fb880df6 admin admin] The token cache is being invalidate because project 0658ab86ba2c4fb8a8b24ace9aaa66b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:39:52.569723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-261e1726-2dab-4ec6-98ca-1326fb880df6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:52.570900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-261e1726-2dab-4ec6-98ca-1326fb880df6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:52.574379 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 873/1748] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:39:52 2022] DELETE /identity/v3/projects/0658ab86ba2c4fb8a8b24ace9aaa66b0 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:52.581213 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca36c4-d33b-4a5e-9e8a-364c22cd3c34 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.581750 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca36c4-d33b-4a5e-9e8a-364c22cd3c34 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.582333 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca36c4-d33b-4a5e-9e8a-364c22cd3c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.613149 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-c7ca36c4-d33b-4a5e-9e8a-364c22cd3c34 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:52.616306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c7ca36c4-d33b-4a5e-9e8a-364c22cd3c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:52.662132 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0583f-de5d-4878-b8e7-e4966d6b40cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:52.662854 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0583f-de5d-4878-b8e7-e4966d6b40cc admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.663156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0583f-de5d-4878-b8e7-e4966d6b40cc admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.663435 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0583f-de5d-4878-b8e7-e4966d6b40cc admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.664492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0583f-de5d-4878-b8e7-e4966d6b40cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:52.665892 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0583f-de5d-4878-b8e7-e4966d6b40cc admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:52.683291 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 876/1749] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:52.700343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0583f-de5d-4878-b8e7-e4966d6b40cc admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:52.706792 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-4fc0583f-de5d-4878-b8e7-e4966d6b40cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:52.717196 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 874/1750] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:39:52.726400 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61923d0c-f92b-4bde-b281-c5ce7df879d0 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.726977 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61923d0c-f92b-4bde-b281-c5ce7df879d0 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.727405 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61923d0c-f92b-4bde-b281-c5ce7df879d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.756863 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0583f-de5d-4878-b8e7-e4966d6b40cc admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:52.767607 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8ee0b-64d7-43dc-91ad-521ea697dbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:52.772119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8ee0b-64d7-43dc-91ad-521ea697dbce admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.773399 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8ee0b-64d7-43dc-91ad-521ea697dbce admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.774151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8ee0b-64d7-43dc-91ad-521ea697dbce admin admin] PATH_INFO: `/v3/projects/41665eb634e24e21ae2ca7cc6a87f8a2/users/df4a8845aee44b2f973bd8ad45a0c027/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.776797 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-61923d0c-f92b-4bde-b281-c5ce7df879d0 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:52.778852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-61923d0c-f92b-4bde-b281-c5ce7df879d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:52.794430 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 875/1751] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:52.802960 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61923d0c-f92b-4bde-b281-c5ce7df879d0 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:52.818650 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8ee0b-64d7-43dc-91ad-521ea697dbce admin admin] RBAC: Authorizing `identity:create_grant(project_id=41665eb634e24e21ae2ca7cc6a87f8a2, user_id=df4a8845aee44b2f973bd8ad45a0c027, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:52.819336 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61923d0c-f92b-4bde-b281-c5ce7df879d0 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:52.824747 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f165d28-ee6f-44f3-a9f8-b0306e5cdfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:52.825921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f165d28-ee6f-44f3-a9f8-b0306e5cdfba admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.826283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f165d28-ee6f-44f3-a9f8-b0306e5cdfba admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.826391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8ee0b-64d7-43dc-91ad-521ea697dbce admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:52.826962 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f165d28-ee6f-44f3-a9f8-b0306e5cdfba admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.827833 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f165d28-ee6f-44f3-a9f8-b0306e5cdfba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:52.828218 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f165d28-ee6f-44f3-a9f8-b0306e5cdfba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:52.830030 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f165d28-ee6f-44f3-a9f8-b0306e5cdfba admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:52.851641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 876/1752] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:52.853323 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 877/1753] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:52 2022] PUT /identity/v3/projects/41665eb634e24e21ae2ca7cc6a87f8a2/users/df4a8845aee44b2f973bd8ad45a0c027/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:52.864512 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f165d28-ee6f-44f3-a9f8-b0306e5cdfba admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:52.872462 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f9b70-7202-494f-b1f1-9ee74a17250d None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.872462 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f9b70-7202-494f-b1f1-9ee74a17250d None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.872656 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f9b70-7202-494f-b1f1-9ee74a17250d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.878303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f165d28-ee6f-44f3-a9f8-b0306e5cdfba admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:52.967088 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-5c7f9b70-7202-494f-b1f1-9ee74a17250d None None] MFA Rules not processed for user `df4a8845aee44b2f973bd8ad45a0c027`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:52.969185 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-5c7f9b70-7202-494f-b1f1-9ee74a17250d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:53.000815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e694ebd-595e-48be-894f-55b872a68525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:53.001692 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e694ebd-595e-48be-894f-55b872a68525 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:53.002087 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e694ebd-595e-48be-894f-55b872a68525 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:53.002457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e694ebd-595e-48be-894f-55b872a68525 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:53.003413 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e694ebd-595e-48be-894f-55b872a68525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:53.005851 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e694ebd-595e-48be-894f-55b872a68525 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:53.043633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:53.060572 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 877/1754] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:52 2022] POST /identity/v3/projects => generated 367 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:53.070161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e694ebd-595e-48be-894f-55b872a68525 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:53.158446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 878/1755] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:53.163709 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f9b70-7202-494f-b1f1-9ee74a17250d None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:53.182963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f9b70-7202-494f-b1f1-9ee74a17250d None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:53.191507 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e694ebd-595e-48be-894f-55b872a68525 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:53.196351 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f240f-1845-4b73-ab92-331dfc35b9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:53.197028 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f240f-1845-4b73-ab92-331dfc35b9f1 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:53.197337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f240f-1845-4b73-ab92-331dfc35b9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:53.197615 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f240f-1845-4b73-ab92-331dfc35b9f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:53.198869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f240f-1845-4b73-ab92-331dfc35b9f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:53.201807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f240f-1845-4b73-ab92-331dfc35b9f1 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:53.204930 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22552e40-d68e-485a-9e63-4fd6a2a990ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:53.209347 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22552e40-d68e-485a-9e63-4fd6a2a990ff admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:53.209857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22552e40-d68e-485a-9e63-4fd6a2a990ff admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:53.210347 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22552e40-d68e-485a-9e63-4fd6a2a990ff admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:53.211447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22552e40-d68e-485a-9e63-4fd6a2a990ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:53.215449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22552e40-d68e-485a-9e63-4fd6a2a990ff admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:53.259517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 879/1756] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:53 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:53.267405 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f240f-1845-4b73-ab92-331dfc35b9f1 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:53.285095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 878/1757] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:53 2022] POST /identity/v3/users => generated 284 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:53.286418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22552e40-d68e-485a-9e63-4fd6a2a990ff admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:53.302355 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22552e40-d68e-485a-9e63-4fd6a2a990ff admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:53.403222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:53.455090 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7207b18-30a0-4ef9-8685-a765afdb9cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:53.456089 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7207b18-30a0-4ef9-8685-a765afdb9cac admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:53.456533 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7207b18-30a0-4ef9-8685-a765afdb9cac admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:53.457212 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7207b18-30a0-4ef9-8685-a765afdb9cac admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:53.458706 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7207b18-30a0-4ef9-8685-a765afdb9cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:53.460737 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7207b18-30a0-4ef9-8685-a765afdb9cac admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:53.491341 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 879/1758] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:53.496848 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7207b18-30a0-4ef9-8685-a765afdb9cac admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:53.504482 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f240f-1845-4b73-ab92-331dfc35b9f1 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:53.516861 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ea755-694b-4a91-9a7d-742494c214b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:53.518170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ea755-694b-4a91-9a7d-742494c214b6 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:53.518551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ea755-694b-4a91-9a7d-742494c214b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:53.519473 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ea755-694b-4a91-9a7d-742494c214b6 admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:53.520739 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ea755-694b-4a91-9a7d-742494c214b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:53.521341 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7207b18-30a0-4ef9-8685-a765afdb9cac admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:53.527196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ea755-694b-4a91-9a7d-742494c214b6 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:53.536277 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e977798-a555-476f-87ca-7580960b5b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:53.537097 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e977798-a555-476f-87ca-7580960b5b03 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:53.537452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e977798-a555-476f-87ca-7580960b5b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:53.537793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e977798-a555-476f-87ca-7580960b5b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:53.538617 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e977798-a555-476f-87ca-7580960b5b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:53.540551 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e977798-a555-476f-87ca-7580960b5b03 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:53.567027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 880/1759] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:53.573837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 880/1760] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:53 2022] POST /identity/v3/users => generated 294 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:53.574070 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e977798-a555-476f-87ca-7580960b5b03 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:53.575162 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ea755-694b-4a91-9a7d-742494c214b6 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:53.593823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e977798-a555-476f-87ca-7580960b5b03 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:53.596872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ea755-694b-4a91-9a7d-742494c214b6 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:53.606750 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd72f3-2b64-4ea1-a8a5-ec55a124348f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:53.607872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a084d0-4486-483b-956c-d3a0af3da8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:53.608740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a084d0-4486-483b-956c-d3a0af3da8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:53.609748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd72f3-2b64-4ea1-a8a5-ec55a124348f admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:53.610659 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a084d0-4486-483b-956c-d3a0af3da8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:53.610659 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a084d0-4486-483b-956c-d3a0af3da8d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:53.611525 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a084d0-4486-483b-956c-d3a0af3da8d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:53.611825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd72f3-2b64-4ea1-a8a5-ec55a124348f admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:53.612191 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd72f3-2b64-4ea1-a8a5-ec55a124348f admin admin] PATH_INFO: `/v3/projects/2abc3018cace459a99c3bae89f4dffc5/users/567252027aff4ec5803f0e143d97e67c/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:53.614909 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a084d0-4486-483b-956c-d3a0af3da8d4 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:53.653468 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd72f3-2b64-4ea1-a8a5-ec55a124348f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2abc3018cace459a99c3bae89f4dffc5, user_id=567252027aff4ec5803f0e143d97e67c, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:53.656545 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 881/1761] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:53.657384 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd72f3-2b64-4ea1-a8a5-ec55a124348f admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:53.659928 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a084d0-4486-483b-956c-d3a0af3da8d4 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:53.701483 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a084d0-4486-483b-956c-d3a0af3da8d4 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:53.715331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 881/1762] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:53 2022] PUT /identity/v3/projects/2abc3018cace459a99c3bae89f4dffc5/users/567252027aff4ec5803f0e143d97e67c/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:53.726413 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c2ee7-8dfc-46ff-8e4d-d88650932d56 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:53.726671 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c2ee7-8dfc-46ff-8e4d-d88650932d56 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:53.727112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c2ee7-8dfc-46ff-8e4d-d88650932d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:53.774793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-423c2ee7-8dfc-46ff-8e4d-d88650932d56 None None] MFA Rules not processed for user `567252027aff4ec5803f0e143d97e67c`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:53.778912 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-423c2ee7-8dfc-46ff-8e4d-d88650932d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:53.890306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:53.918680 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd1914-522b-4ee0-a23d-d3dbd1a8d389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:53.921999 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd1914-522b-4ee0-a23d-d3dbd1a8d389 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:53.922845 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd1914-522b-4ee0-a23d-d3dbd1a8d389 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:53.923630 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd1914-522b-4ee0-a23d-d3dbd1a8d389 admin admin] PATH_INFO: `/v3/projects/db1409067fb8471894cc2cd25e498a98/users/fed989144e1449fb9b8d2b5352407b7f/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:53.971903 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 882/1763] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:53 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:53.978865 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c2ee7-8dfc-46ff-8e4d-d88650932d56 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:53.987816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd1914-522b-4ee0-a23d-d3dbd1a8d389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db1409067fb8471894cc2cd25e498a98, user_id=fed989144e1449fb9b8d2b5352407b7f, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:53.991254 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd1914-522b-4ee0-a23d-d3dbd1a8d389 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:54.009234 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c2ee7-8dfc-46ff-8e4d-d88650932d56 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:54.011930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 882/1764] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:53 2022] PUT /identity/v3/projects/db1409067fb8471894cc2cd25e498a98/users/fed989144e1449fb9b8d2b5352407b7f/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:54.018188 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd1914-522b-4ee0-a23d-d3dbd1a8d389 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:54.115824 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54e8d9-a8ee-441e-99a4-4fb483b2a127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:54.116517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54e8d9-a8ee-441e-99a4-4fb483b2a127 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.116801 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54e8d9-a8ee-441e-99a4-4fb483b2a127 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.117099 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54e8d9-a8ee-441e-99a4-4fb483b2a127 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.117862 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54e8d9-a8ee-441e-99a4-4fb483b2a127 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:54.120006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54e8d9-a8ee-441e-99a4-4fb483b2a127 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:54.139265 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd1914-522b-4ee0-a23d-d3dbd1a8d389 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:54.147636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4defc-dff5-455c-9339-75621e7209c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:54.148654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4defc-dff5-455c-9339-75621e7209c9 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.149115 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4defc-dff5-455c-9339-75621e7209c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.149549 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4defc-dff5-455c-9339-75621e7209c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.151246 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4defc-dff5-455c-9339-75621e7209c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:54.153447 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4defc-dff5-455c-9339-75621e7209c9 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:54.160097 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 883/1765] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:53 2022] POST /identity/v3/projects => generated 369 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:54.162459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54e8d9-a8ee-441e-99a4-4fb483b2a127 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:54.172146 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 883/1766] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:54.173309 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4defc-dff5-455c-9339-75621e7209c9 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:54.187086 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4defc-dff5-455c-9339-75621e7209c9 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:54.229873 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:54.252231 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54e8d9-a8ee-441e-99a4-4fb483b2a127 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:54.272547 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59d852-fdd9-4383-8bf2-c4d9e3266010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:54.273681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59d852-fdd9-4383-8bf2-c4d9e3266010 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.274069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59d852-fdd9-4383-8bf2-c4d9e3266010 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.274621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59d852-fdd9-4383-8bf2-c4d9e3266010 admin admin] PATH_INFO: `/v3/projects/db1409067fb8471894cc2cd25e498a98/users/fed989144e1449fb9b8d2b5352407b7f/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.277328 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59d852-fdd9-4383-8bf2-c4d9e3266010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db1409067fb8471894cc2cd25e498a98, user_id=fed989144e1449fb9b8d2b5352407b7f, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:54.277589 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b9cce-173f-429a-957b-cb8bb7bf41b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:54.278140 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b9cce-173f-429a-957b-cb8bb7bf41b6 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.278453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b9cce-173f-429a-957b-cb8bb7bf41b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.278895 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b9cce-173f-429a-957b-cb8bb7bf41b6 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.279628 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b9cce-173f-429a-957b-cb8bb7bf41b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:54.280693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59d852-fdd9-4383-8bf2-c4d9e3266010 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:54.281861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b9cce-173f-429a-957b-cb8bb7bf41b6 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:54.313125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 884/1767] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:54 2022] PUT /identity/v3/projects/db1409067fb8471894cc2cd25e498a98/users/fed989144e1449fb9b8d2b5352407b7f/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:39:54.324629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35968693-e536-4d2c-9487-ad0a20a268cf None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.325148 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35968693-e536-4d2c-9487-ad0a20a268cf None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.326453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35968693-e536-4d2c-9487-ad0a20a268cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.341047 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 884/1768] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:54 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:39:54.342943 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b9cce-173f-429a-957b-cb8bb7bf41b6 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:54.363772 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b9cce-173f-429a-957b-cb8bb7bf41b6 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:54.373289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-35968693-e536-4d2c-9487-ad0a20a268cf None None] MFA Rules not processed for user `fed989144e1449fb9b8d2b5352407b7f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:54.374852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-35968693-e536-4d2c-9487-ad0a20a268cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:54.442727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:54.465236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc5649-0fd0-4b94-8f4f-799e49803f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:54.466168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc5649-0fd0-4b94-8f4f-799e49803f0c admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.466597 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc5649-0fd0-4b94-8f4f-799e49803f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.467276 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc5649-0fd0-4b94-8f4f-799e49803f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.468480 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc5649-0fd0-4b94-8f4f-799e49803f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:54.470687 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc5649-0fd0-4b94-8f4f-799e49803f0c admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:54.486749 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 885/1769] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:54.495621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 885/1770] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:54.497891 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc5649-0fd0-4b94-8f4f-799e49803f0c admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:54.509865 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35968693-e536-4d2c-9487-ad0a20a268cf None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:54.511306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc5649-0fd0-4b94-8f4f-799e49803f0c admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:54.511636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-35968693-e536-4d2c-9487-ad0a20a268cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:54.524103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797179e1-e865-4532-a5ff-2c63652dba64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:54.525876 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797179e1-e865-4532-a5ff-2c63652dba64 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.527735 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797179e1-e865-4532-a5ff-2c63652dba64 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.528567 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797179e1-e865-4532-a5ff-2c63652dba64 admin admin] PATH_INFO: `/v3/projects/cef1b4ce3d1e4dd48fce791682b1f8b0/users/40b8b03b013348a385de2fd1626e941b/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.574841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797179e1-e865-4532-a5ff-2c63652dba64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cef1b4ce3d1e4dd48fce791682b1f8b0, user_id=40b8b03b013348a385de2fd1626e941b, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:54.578848 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797179e1-e865-4532-a5ff-2c63652dba64 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:54.598143 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 886/1771] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:54 2022] PUT /identity/v3/projects/cef1b4ce3d1e4dd48fce791682b1f8b0/users/40b8b03b013348a385de2fd1626e941b/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:54.602438 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797179e1-e865-4532-a5ff-2c63652dba64 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:54.618063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35968693-e536-4d2c-9487-ad0a20a268cf None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:54.686612 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e971117-9f4c-47ac-9cf8-d44c452b168d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:54.687488 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e971117-9f4c-47ac-9cf8-d44c452b168d service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.688059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e971117-9f4c-47ac-9cf8-d44c452b168d service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.688516 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e971117-9f4c-47ac-9cf8-d44c452b168d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.699753 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797179e1-e865-4532-a5ff-2c63652dba64 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:54.707342 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9314b-baa7-4aa8-a646-99b70b5fd910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:54.708178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9314b-baa7-4aa8-a646-99b70b5fd910 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.708516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9314b-baa7-4aa8-a646-99b70b5fd910 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.708824 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9314b-baa7-4aa8-a646-99b70b5fd910 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.709710 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede9314b-baa7-4aa8-a646-99b70b5fd910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:54.712032 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede9314b-baa7-4aa8-a646-99b70b5fd910 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:54.712203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:54.723863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 887/1772] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:54.728316 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9314b-baa7-4aa8-a646-99b70b5fd910 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:54.741052 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9314b-baa7-4aa8-a646-99b70b5fd910 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:54.747040 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884cc1e-fbe1-4343-a662-98ee0bad30f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:54.748031 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884cc1e-fbe1-4343-a662-98ee0bad30f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.748210 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884cc1e-fbe1-4343-a662-98ee0bad30f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.748487 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884cc1e-fbe1-4343-a662-98ee0bad30f2 admin admin] PATH_INFO: `/v3/projects/cef1b4ce3d1e4dd48fce791682b1f8b0/users/40b8b03b013348a385de2fd1626e941b/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.750554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884cc1e-fbe1-4343-a662-98ee0bad30f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cef1b4ce3d1e4dd48fce791682b1f8b0, user_id=40b8b03b013348a385de2fd1626e941b, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:54.753259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884cc1e-fbe1-4343-a662-98ee0bad30f2 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:54.764451 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e971117-9f4c-47ac-9cf8-d44c452b168d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:54.764852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 888/1773] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:54 2022] PUT /identity/v3/projects/cef1b4ce3d1e4dd48fce791682b1f8b0/users/40b8b03b013348a385de2fd1626e941b/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:39:54.766435 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e971117-9f4c-47ac-9cf8-d44c452b168d service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:54.775792 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b48e59-4baa-4847-a7f9-5425fa1cf5b8 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.776215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b48e59-4baa-4847-a7f9-5425fa1cf5b8 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.776591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b48e59-4baa-4847-a7f9-5425fa1cf5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.777335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 886/1774] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:39:54.810111 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-43b48e59-4baa-4847-a7f9-5425fa1cf5b8 None None] MFA Rules not processed for user `40b8b03b013348a385de2fd1626e941b`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:54.811502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-43b48e59-4baa-4847-a7f9-5425fa1cf5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:54.945861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 889/1775] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:54 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:55.094006 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec2bfa-1922-47f7-8fa3-102aea111bf9 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:55.094877 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec2bfa-1922-47f7-8fa3-102aea111bf9 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:55.094877 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec2bfa-1922-47f7-8fa3-102aea111bf9 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:55.096835 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 887/1776] 10.222.0.71 () {54 vars in 921 bytes} [Thu Jul 21 20:39:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:55.144300 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3035c8-1007-44d2-9667-0ff02feec699 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:55.144997 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3035c8-1007-44d2-9667-0ff02feec699 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:55.145573 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3035c8-1007-44d2-9667-0ff02feec699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:55.180795 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-6f3035c8-1007-44d2-9667-0ff02feec699 None None] MFA Rules not processed for user `364e80ec79b94812a1df76c62c2b13a0`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:55.184466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-6f3035c8-1007-44d2-9667-0ff02feec699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:55.233194 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:55.285452 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec2bfa-1922-47f7-8fa3-102aea111bf9 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:55.288023 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-64ec2bfa-1922-47f7-8fa3-102aea111bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:55.335907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 890/1777] 10.222.0.71 () {58 vars in 1018 bytes} [Thu Jul 21 20:39:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:55.422657 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec2bfa-1922-47f7-8fa3-102aea111bf9 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:55.429633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a90da-905d-4b22-b150-dd7f9344f021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:55.430866 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a90da-905d-4b22-b150-dd7f9344f021 service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:55.431377 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a90da-905d-4b22-b150-dd7f9344f021 service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:55.433380 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a90da-905d-4b22-b150-dd7f9344f021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:55.451034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a90da-905d-4b22-b150-dd7f9344f021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:55.453339 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a90da-905d-4b22-b150-dd7f9344f021 service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:55.483852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 888/1778] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:55 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:55.514197 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3035c8-1007-44d2-9667-0ff02feec699 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:55.516053 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-6f3035c8-1007-44d2-9667-0ff02feec699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:55.532919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3035c8-1007-44d2-9667-0ff02feec699 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:55.546782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a90da-905d-4b22-b150-dd7f9344f021 service cinder] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:55.556983 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a90da-905d-4b22-b150-dd7f9344f021 service cinder] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:55.561711 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05415fd0-bbdf-40d2-8eb5-0b132588eb5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:55.562694 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05415fd0-bbdf-40d2-8eb5-0b132588eb5e service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:55.563153 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05415fd0-bbdf-40d2-8eb5-0b132588eb5e service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:55.564300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05415fd0-bbdf-40d2-8eb5-0b132588eb5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:55.573188 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdca79-865f-4ad2-bfdf-9012c357c368 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '364e80ec79b94812a1df76c62c2b13a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:55.574749 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdca79-865f-4ad2-bfdf-9012c357c368 service placement] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:55.575224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdca79-865f-4ad2-bfdf-9012c357c368 service placement] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:55.575722 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdca79-865f-4ad2-bfdf-9012c357c368 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:55.587200 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdca79-865f-4ad2-bfdf-9012c357c368 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:55.605954 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdca79-865f-4ad2-bfdf-9012c357c368 service placement] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:55.636541 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 891/1779] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:39:55.673306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05415fd0-bbdf-40d2-8eb5-0b132588eb5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:55.678310 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05415fd0-bbdf-40d2-8eb5-0b132588eb5e service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:55.689252 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 889/1780] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:39:55 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:39:56.535952 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdca79-865f-4ad2-bfdf-9012c357c368 service placement] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:56.537563 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c9cdca79-865f-4ad2-bfdf-9012c357c368 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:56.631302 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdca79-865f-4ad2-bfdf-9012c357c368 service placement] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:56.636153 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ac695-cac9-4560-949d-f70888e02fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:56.637052 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ac695-cac9-4560-949d-f70888e02fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:56.637891 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ac695-cac9-4560-949d-f70888e02fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:56.637891 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ac695-cac9-4560-949d-f70888e02fc0 admin admin] PATH_INFO: `/v3/users/53b97bb92bff496688e900b4e22bb258` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:56.638929 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ac695-cac9-4560-949d-f70888e02fc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53b97bb92bff496688e900b4e22bb258)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:56.640903 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ac695-cac9-4560-949d-f70888e02fc0 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:56.698213 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4b9ac695-cac9-4560-949d-f70888e02fc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53b97bb92bff496688e900b4e22bb258'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:56.706171 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4b9ac695-cac9-4560-949d-f70888e02fc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53b97bb92bff496688e900b4e22bb258'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:56.706829 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4b9ac695-cac9-4560-949d-f70888e02fc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53b97bb92bff496688e900b4e22bb258'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:56.721792 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4b9ac695-cac9-4560-949d-f70888e02fc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53b97bb92bff496688e900b4e22bb258'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:56.729777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4b9ac695-cac9-4560-949d-f70888e02fc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53b97bb92bff496688e900b4e22bb258'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:56.737334 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 892/1781] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:39:56 2022] DELETE /identity/v3/users/53b97bb92bff496688e900b4e22bb258 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:57.137508 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05415fd0-bbdf-40d2-8eb5-0b132588eb5e service cinder] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:57.161601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05415fd0-bbdf-40d2-8eb5-0b132588eb5e service cinder] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:57.283669 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a69116-6d60-4ce5-be53-28636009d172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:57.284609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a69116-6d60-4ce5-be53-28636009d172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:57.285051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a69116-6d60-4ce5-be53-28636009d172 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:57.285365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a69116-6d60-4ce5-be53-28636009d172 admin admin] PATH_INFO: `/v3/projects/80586e6c15ea4983b2606837706fa71f` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:57.286709 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a69116-6d60-4ce5-be53-28636009d172 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80586e6c15ea4983b2606837706fa71f)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:57.288674 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a69116-6d60-4ce5-be53-28636009d172 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:57.292107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:57.367700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-26a69116-6d60-4ce5-be53-28636009d172 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80586e6c15ea4983b2606837706fa71f', 'request_id': 'req-26a69116-6d60-4ce5-be53-28636009d172'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:57.386406 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-26a69116-6d60-4ce5-be53-28636009d172 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80586e6c15ea4983b2606837706fa71f', 'request_id': 'req-26a69116-6d60-4ce5-be53-28636009d172'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:57.403970 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-26a69116-6d60-4ce5-be53-28636009d172 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80586e6c15ea4983b2606837706fa71f', 'request_id': 'req-26a69116-6d60-4ce5-be53-28636009d172'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:57.435437 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-26a69116-6d60-4ce5-be53-28636009d172 admin admin] The token cache is being invalidate because project 80586e6c15ea4983b2606837706fa71f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:39:57.435437 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-26a69116-6d60-4ce5-be53-28636009d172 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:57.435437 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-26a69116-6d60-4ce5-be53-28636009d172 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:57.436818 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 890/1782] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:39:57 2022] DELETE /identity/v3/projects/80586e6c15ea4983b2606837706fa71f => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:57.441006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ac695-cac9-4560-949d-f70888e02fc0 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:57.443303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-4b9ac695-cac9-4560-949d-f70888e02fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:57.468002 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ac695-cac9-4560-949d-f70888e02fc0 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:57.522457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:57.557957 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c46bc3-4a44-4bc9-b54c-9b8d05f32e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:57.558874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c46bc3-4a44-4bc9-b54c-9b8d05f32e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:57.559163 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c46bc3-4a44-4bc9-b54c-9b8d05f32e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:57.559661 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c46bc3-4a44-4bc9-b54c-9b8d05f32e76 admin admin] PATH_INFO: `/v3/users/19551e1cc3ce406dbb565de2516c3b9f` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:57.562425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c46bc3-4a44-4bc9-b54c-9b8d05f32e76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19551e1cc3ce406dbb565de2516c3b9f)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:57.564689 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c46bc3-4a44-4bc9-b54c-9b8d05f32e76 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:57.650230 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-b5c46bc3-4a44-4bc9-b54c-9b8d05f32e76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19551e1cc3ce406dbb565de2516c3b9f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:57.662143 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-b5c46bc3-4a44-4bc9-b54c-9b8d05f32e76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19551e1cc3ce406dbb565de2516c3b9f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:57.662846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-b5c46bc3-4a44-4bc9-b54c-9b8d05f32e76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19551e1cc3ce406dbb565de2516c3b9f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:57.703706 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-b5c46bc3-4a44-4bc9-b54c-9b8d05f32e76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19551e1cc3ce406dbb565de2516c3b9f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:57.717436 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-b5c46bc3-4a44-4bc9-b54c-9b8d05f32e76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19551e1cc3ce406dbb565de2516c3b9f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:57.733262 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 893/1783] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:39:57 2022] DELETE /identity/v3/users/19551e1cc3ce406dbb565de2516c3b9f => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:39:58.118900 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a69116-6d60-4ce5-be53-28636009d172 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:58.121180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-26a69116-6d60-4ce5-be53-28636009d172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:58.136383 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a69116-6d60-4ce5-be53-28636009d172 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:58.222821 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23157a1d-498f-4ad4-bc16-b9d81afc2c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:58.223968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23157a1d-498f-4ad4-bc16-b9d81afc2c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:58.223968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23157a1d-498f-4ad4-bc16-b9d81afc2c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:58.224264 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23157a1d-498f-4ad4-bc16-b9d81afc2c48 admin admin] PATH_INFO: `/v3/projects/0cae91ea906b4b09a6bcc3cb3545a078` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:58.225753 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23157a1d-498f-4ad4-bc16-b9d81afc2c48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cae91ea906b4b09a6bcc3cb3545a078)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:58.228573 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23157a1d-498f-4ad4-bc16-b9d81afc2c48 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:58.285985 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-23157a1d-498f-4ad4-bc16-b9d81afc2c48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cae91ea906b4b09a6bcc3cb3545a078', 'request_id': 'req-23157a1d-498f-4ad4-bc16-b9d81afc2c48'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:58.295993 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-23157a1d-498f-4ad4-bc16-b9d81afc2c48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cae91ea906b4b09a6bcc3cb3545a078', 'request_id': 'req-23157a1d-498f-4ad4-bc16-b9d81afc2c48'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:58.305887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-23157a1d-498f-4ad4-bc16-b9d81afc2c48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cae91ea906b4b09a6bcc3cb3545a078', 'request_id': 'req-23157a1d-498f-4ad4-bc16-b9d81afc2c48'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:58.326371 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-23157a1d-498f-4ad4-bc16-b9d81afc2c48 admin admin] The token cache is being invalidate because project 0cae91ea906b4b09a6bcc3cb3545a078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:39:58.326720 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-23157a1d-498f-4ad4-bc16-b9d81afc2c48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:58.327732 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-23157a1d-498f-4ad4-bc16-b9d81afc2c48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:39:58.330489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 891/1784] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:39:58 2022] DELETE /identity/v3/projects/0cae91ea906b4b09a6bcc3cb3545a078 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:58.338958 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b71369-d8bc-4195-9be4-bfb082a49000 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:58.338958 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b71369-d8bc-4195-9be4-bfb082a49000 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:58.338958 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b71369-d8bc-4195-9be4-bfb082a49000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:58.358237 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-56b71369-d8bc-4195-9be4-bfb082a49000 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:58.359485 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-56b71369-d8bc-4195-9be4-bfb082a49000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:58.446371 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 894/1785] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:39:58.454569 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2222f-214f-48f3-87af-2f79ae246454 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:58.454935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2222f-214f-48f3-87af-2f79ae246454 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:58.455274 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2222f-214f-48f3-87af-2f79ae246454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:58.486445 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-27b2222f-214f-48f3-87af-2f79ae246454 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:58.487743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-27b2222f-214f-48f3-87af-2f79ae246454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:58.497497 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 892/1786] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:58.502143 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b71369-d8bc-4195-9be4-bfb082a49000 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:58.513976 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b71369-d8bc-4195-9be4-bfb082a49000 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:58.519769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96bd536-0411-4621-80b8-36b519071162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:58.520673 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96bd536-0411-4621-80b8-36b519071162 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:58.521043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96bd536-0411-4621-80b8-36b519071162 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:58.521523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96bd536-0411-4621-80b8-36b519071162 admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:58.522513 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96bd536-0411-4621-80b8-36b519071162 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:58.522881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96bd536-0411-4621-80b8-36b519071162 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:58.524902 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96bd536-0411-4621-80b8-36b519071162 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:58.546399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 895/1787] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:39:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:58.552403 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2222f-214f-48f3-87af-2f79ae246454 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:58.563365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2222f-214f-48f3-87af-2f79ae246454 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:58.568268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43790e3-b01c-4c51-b080-e3cb547eeab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:58.568975 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43790e3-b01c-4c51-b080-e3cb547eeab0 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:58.569304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43790e3-b01c-4c51-b080-e3cb547eeab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:58.569580 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43790e3-b01c-4c51-b080-e3cb547eeab0 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:58.570336 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43790e3-b01c-4c51-b080-e3cb547eeab0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:58.572531 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43790e3-b01c-4c51-b080-e3cb547eeab0 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:58.588500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:58.605874 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 893/1788] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:58 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:58.610105 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96bd536-0411-4621-80b8-36b519071162 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:58.621584 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96bd536-0411-4621-80b8-36b519071162 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:58.686486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:58.737013 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a66297-fa80-459b-9854-1033442e1485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:58.752829 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a66297-fa80-459b-9854-1033442e1485 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:58.753586 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a66297-fa80-459b-9854-1033442e1485 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:58.755855 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a66297-fa80-459b-9854-1033442e1485 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:58.758004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a66297-fa80-459b-9854-1033442e1485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:58.777880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a66297-fa80-459b-9854-1033442e1485 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:58.818732 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 896/1789] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:58 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:58.825253 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43790e3-b01c-4c51-b080-e3cb547eeab0 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:58.851491 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43790e3-b01c-4c51-b080-e3cb547eeab0 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:58.872506 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a1c5f-03b5-450e-a6f2-91fdc3da1af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:58.874029 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a1c5f-03b5-450e-a6f2-91fdc3da1af7 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:58.874567 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a1c5f-03b5-450e-a6f2-91fdc3da1af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:58.875225 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a1c5f-03b5-450e-a6f2-91fdc3da1af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:58.878813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a1c5f-03b5-450e-a6f2-91fdc3da1af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:58.884285 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a1c5f-03b5-450e-a6f2-91fdc3da1af7 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:58.943524 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 894/1790] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:58.944071 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a66297-fa80-459b-9854-1033442e1485 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:58.963455 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a66297-fa80-459b-9854-1033442e1485 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:59.005177 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381defa-4dd6-42dd-979f-f15a73e05a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:59.005889 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381defa-4dd6-42dd-979f-f15a73e05a62 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:59.006179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381defa-4dd6-42dd-979f-f15a73e05a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:59.006462 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381defa-4dd6-42dd-979f-f15a73e05a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:59.007224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381defa-4dd6-42dd-979f-f15a73e05a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:59.008939 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381defa-4dd6-42dd-979f-f15a73e05a62 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:59.046178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 897/1791] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:59.047704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a1c5f-03b5-450e-a6f2-91fdc3da1af7 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:59.075991 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a1c5f-03b5-450e-a6f2-91fdc3da1af7 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:59.083496 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556729e9-73f3-47cf-bbd6-fa3a498d5424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:59.084740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556729e9-73f3-47cf-bbd6-fa3a498d5424 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:59.085296 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556729e9-73f3-47cf-bbd6-fa3a498d5424 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:59.085729 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556729e9-73f3-47cf-bbd6-fa3a498d5424 admin admin] PATH_INFO: `/v3/projects/2378990540cc490aa181b4201cc1b37f/users/237264a4bee44a9cb83a9b57d31605d0/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:59.115521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556729e9-73f3-47cf-bbd6-fa3a498d5424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2378990540cc490aa181b4201cc1b37f, user_id=237264a4bee44a9cb83a9b57d31605d0, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:59.123489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556729e9-73f3-47cf-bbd6-fa3a498d5424 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:59.158783 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 895/1792] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:59 2022] PUT /identity/v3/projects/2378990540cc490aa181b4201cc1b37f/users/237264a4bee44a9cb83a9b57d31605d0/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:59.160007 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214d65b-425f-4b9a-b01a-04f5bc73f3cd None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:59.160304 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214d65b-425f-4b9a-b01a-04f5bc73f3cd None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:59.160588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214d65b-425f-4b9a-b01a-04f5bc73f3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:59.228169 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-e214d65b-425f-4b9a-b01a-04f5bc73f3cd None None] MFA Rules not processed for user `237264a4bee44a9cb83a9b57d31605d0`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:59.230191 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-e214d65b-425f-4b9a-b01a-04f5bc73f3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:59.375696 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 898/1793] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:39:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:39:59.379802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556729e9-73f3-47cf-bbd6-fa3a498d5424 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:59.529768 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556729e9-73f3-47cf-bbd6-fa3a498d5424 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:59.542711 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8ebd0-29a9-425f-9adf-51b2f0f3cff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:59.543517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8ebd0-29a9-425f-9adf-51b2f0f3cff0 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:59.543828 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8ebd0-29a9-425f-9adf-51b2f0f3cff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:59.544128 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8ebd0-29a9-425f-9adf-51b2f0f3cff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:59.544868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8ebd0-29a9-425f-9adf-51b2f0f3cff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:59.550596 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8ebd0-29a9-425f-9adf-51b2f0f3cff0 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:59.605463 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 896/1794] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:39:59 2022] POST /identity/v3/projects => generated 387 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:39:59.611959 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214d65b-425f-4b9a-b01a-04f5bc73f3cd None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:59.633746 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214d65b-425f-4b9a-b01a-04f5bc73f3cd None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:59.813788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a933e-964b-40d8-8225-7de488ffea45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:59.814978 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a933e-964b-40d8-8225-7de488ffea45 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:59.816981 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a933e-964b-40d8-8225-7de488ffea45 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:59.817688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a933e-964b-40d8-8225-7de488ffea45 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:59.819271 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a933e-964b-40d8-8225-7de488ffea45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:59.830435 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a933e-964b-40d8-8225-7de488ffea45 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:59.862003 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 899/1795] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:39:59 2022] POST /identity/v3/users => generated 300 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:39:59.866100 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8ebd0-29a9-425f-9adf-51b2f0f3cff0 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:59.883348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8ebd0-29a9-425f-9adf-51b2f0f3cff0 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:59.888236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35aa75-1e56-463a-8d45-88f491414b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:59.889218 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35aa75-1e56-463a-8d45-88f491414b52 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:59.889498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35aa75-1e56-463a-8d45-88f491414b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:59.889771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35aa75-1e56-463a-8d45-88f491414b52 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:59.890549 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e35aa75-1e56-463a-8d45-88f491414b52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:59.892148 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e35aa75-1e56-463a-8d45-88f491414b52 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:59.909509 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 897/1796] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:59.910776 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a933e-964b-40d8-8225-7de488ffea45 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:59.922088 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a933e-964b-40d8-8225-7de488ffea45 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:59.927908 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7688589-1a15-4b96-a265-9d963fc7ec79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:59.928961 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7688589-1a15-4b96-a265-9d963fc7ec79 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:59.929524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7688589-1a15-4b96-a265-9d963fc7ec79 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:59.929701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7688589-1a15-4b96-a265-9d963fc7ec79 admin admin] PATH_INFO: `/v3/projects/d00bcd228fac40b49c0b17661b9f601b/users/03c82e90e4254a79abb6be4bebd91a3d/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:59.986990 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7688589-1a15-4b96-a265-9d963fc7ec79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d00bcd228fac40b49c0b17661b9f601b, user_id=03c82e90e4254a79abb6be4bebd91a3d, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:59.989303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35aa75-1e56-463a-8d45-88f491414b52 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:59.990759 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7688589-1a15-4b96-a265-9d963fc7ec79 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:59.992388 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3e35aa75-1e56-463a-8d45-88f491414b52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:00.008043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 900/1797] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:39:59 2022] PUT /identity/v3/projects/d00bcd228fac40b49c0b17661b9f601b/users/03c82e90e4254a79abb6be4bebd91a3d/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:00.012464 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7688589-1a15-4b96-a265-9d963fc7ec79 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:00.037780 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:00.038061 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35aa75-1e56-463a-8d45-88f491414b52 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:00.111491 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b06ec-2e60-4de1-9907-e008070507c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:00.112381 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b06ec-2e60-4de1-9907-e008070507c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.112658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b06ec-2e60-4de1-9907-e008070507c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.112930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b06ec-2e60-4de1-9907-e008070507c7 admin admin] PATH_INFO: `/v3/users/567252027aff4ec5803f0e143d97e67c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.114438 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b06ec-2e60-4de1-9907-e008070507c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=567252027aff4ec5803f0e143d97e67c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:00.117397 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b06ec-2e60-4de1-9907-e008070507c7 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:00.136454 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7688589-1a15-4b96-a265-9d963fc7ec79 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:00.142098 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fffb00-c424-49cb-af7b-9154e2fc99b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:00.142866 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fffb00-c424-49cb-af7b-9154e2fc99b6 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.143241 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fffb00-c424-49cb-af7b-9154e2fc99b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.143539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fffb00-c424-49cb-af7b-9154e2fc99b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.144284 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fffb00-c424-49cb-af7b-9154e2fc99b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:00.146212 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fffb00-c424-49cb-af7b-9154e2fc99b6 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:00.167013 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:00.170306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 901/1798] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:40:00.175899 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fffb00-c424-49cb-af7b-9154e2fc99b6 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:00.184255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2f7b06ec-2e60-4de1-9907-e008070507c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '567252027aff4ec5803f0e143d97e67c'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:00.193541 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2f7b06ec-2e60-4de1-9907-e008070507c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '567252027aff4ec5803f0e143d97e67c'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:00.194218 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2f7b06ec-2e60-4de1-9907-e008070507c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '567252027aff4ec5803f0e143d97e67c'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:00.197168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fffb00-c424-49cb-af7b-9154e2fc99b6 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:00.204257 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2f7b06ec-2e60-4de1-9907-e008070507c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '567252027aff4ec5803f0e143d97e67c'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:00.204988 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2f7b06ec-2e60-4de1-9907-e008070507c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '567252027aff4ec5803f0e143d97e67c'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:00.205625 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb8a78-7221-4833-a29c-825922077376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:00.206660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb8a78-7221-4833-a29c-825922077376 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.207091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb8a78-7221-4833-a29c-825922077376 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.207470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb8a78-7221-4833-a29c-825922077376 admin admin] PATH_INFO: `/v3/projects/d00bcd228fac40b49c0b17661b9f601b/users/03c82e90e4254a79abb6be4bebd91a3d/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.209728 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb8a78-7221-4833-a29c-825922077376 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d00bcd228fac40b49c0b17661b9f601b, user_id=03c82e90e4254a79abb6be4bebd91a3d, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:00.212549 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb8a78-7221-4833-a29c-825922077376 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:00.225143 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 902/1799] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:00 2022] PUT /identity/v3/projects/d00bcd228fac40b49c0b17661b9f601b/users/03c82e90e4254a79abb6be4bebd91a3d/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:00.229954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 898/1800] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:39:59 2022] DELETE /identity/v3/users/567252027aff4ec5803f0e143d97e67c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:00.235775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a75abb-769c-43e0-b521-1e4d99a76d76 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.236069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a75abb-769c-43e0-b521-1e4d99a76d76 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.236352 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a75abb-769c-43e0-b521-1e4d99a76d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.236454 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724b939-ce8e-499e-bdc4-1fa7fe575d38 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.237224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724b939-ce8e-499e-bdc4-1fa7fe575d38 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.237834 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724b939-ce8e-499e-bdc4-1fa7fe575d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.268523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-2724b939-ce8e-499e-bdc4-1fa7fe575d38 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:00.270618 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-2724b939-ce8e-499e-bdc4-1fa7fe575d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:00.270997 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-40a75abb-769c-43e0-b521-1e4d99a76d76 None None] MFA Rules not processed for user `03c82e90e4254a79abb6be4bebd91a3d`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:00.274968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-40a75abb-769c-43e0-b521-1e4d99a76d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:00.373611 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 903/1801] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:00.383066 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 899/1802] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:00.387058 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724b939-ce8e-499e-bdc4-1fa7fe575d38 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:00.390368 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-2724b939-ce8e-499e-bdc4-1fa7fe575d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:00.401699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a75abb-769c-43e0-b521-1e4d99a76d76 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:00.405247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-40a75abb-769c-43e0-b521-1e4d99a76d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:00.493186 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724b939-ce8e-499e-bdc4-1fa7fe575d38 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:00.499227 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e5550-771b-4777-a8e1-7f5a3c05bc07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:00.500226 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e5550-771b-4777-a8e1-7f5a3c05bc07 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.501208 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e5550-771b-4777-a8e1-7f5a3c05bc07 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.501208 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e5550-771b-4777-a8e1-7f5a3c05bc07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.514415 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e5550-771b-4777-a8e1-7f5a3c05bc07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:00.515879 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e5550-771b-4777-a8e1-7f5a3c05bc07 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:00.527228 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a75abb-769c-43e0-b521-1e4d99a76d76 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:00.531749 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801d21c-d8fe-435a-b8cb-c0c98d41c590 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:00.532544 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801d21c-d8fe-435a-b8cb-c0c98d41c590 service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.533157 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801d21c-d8fe-435a-b8cb-c0c98d41c590 service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.533546 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801d21c-d8fe-435a-b8cb-c0c98d41c590 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.537294 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 904/1803] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:40:00.546053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801d21c-d8fe-435a-b8cb-c0c98d41c590 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:00.549306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801d21c-d8fe-435a-b8cb-c0c98d41c590 service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:00.568850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 900/1804] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:00.604843 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e5550-771b-4777-a8e1-7f5a3c05bc07 service neutron] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:00.606938 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-da4e5550-771b-4777-a8e1-7f5a3c05bc07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:00.624717 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e5550-771b-4777-a8e1-7f5a3c05bc07 service neutron] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:00.632830 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321deb0-7cfb-45ac-a556-cd62f723bf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:00.633968 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321deb0-7cfb-45ac-a556-cd62f723bf58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.634105 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321deb0-7cfb-45ac-a556-cd62f723bf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.634351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321deb0-7cfb-45ac-a556-cd62f723bf58 admin admin] PATH_INFO: `/v3/users/237264a4bee44a9cb83a9b57d31605d0` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.637788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321deb0-7cfb-45ac-a556-cd62f723bf58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=237264a4bee44a9cb83a9b57d31605d0)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:00.638475 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321deb0-7cfb-45ac-a556-cd62f723bf58 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:00.688554 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:00.712739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-c321deb0-7cfb-45ac-a556-cd62f723bf58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '237264a4bee44a9cb83a9b57d31605d0'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:00.726667 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-c321deb0-7cfb-45ac-a556-cd62f723bf58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '237264a4bee44a9cb83a9b57d31605d0'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:00.727597 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-c321deb0-7cfb-45ac-a556-cd62f723bf58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '237264a4bee44a9cb83a9b57d31605d0'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:00.746876 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-c321deb0-7cfb-45ac-a556-cd62f723bf58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '237264a4bee44a9cb83a9b57d31605d0'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:00.764624 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-c321deb0-7cfb-45ac-a556-cd62f723bf58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '237264a4bee44a9cb83a9b57d31605d0'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:00.782078 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 905/1805] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:00 2022] DELETE /identity/v3/users/237264a4bee44a9cb83a9b57d31605d0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:00.784777 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af653d-239b-4868-8948-b1568d299ed5 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.785369 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af653d-239b-4868-8948-b1568d299ed5 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.785822 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af653d-239b-4868-8948-b1568d299ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.809204 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-59af653d-239b-4868-8948-b1568d299ed5 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:00.810911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-59af653d-239b-4868-8948-b1568d299ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:00.869803 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:00.932726 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321deb0-7cfb-45ac-a556-cd62f723bf58 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:00.935829 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c321deb0-7cfb-45ac-a556-cd62f723bf58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:00.938862 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 901/1806] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:00.952525 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af653d-239b-4868-8948-b1568d299ed5 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:00.954425 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-59af653d-239b-4868-8948-b1568d299ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:01.019950 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321deb0-7cfb-45ac-a556-cd62f723bf58 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:01.044292 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1d417-55db-4df5-8930-6b71ec2c74dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:01.045849 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1d417-55db-4df5-8930-6b71ec2c74dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:01.046356 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1d417-55db-4df5-8930-6b71ec2c74dd admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:01.046867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1d417-55db-4df5-8930-6b71ec2c74dd admin admin] PATH_INFO: `/v3/projects/2abc3018cace459a99c3bae89f4dffc5` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:01.049974 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db1d417-55db-4df5-8930-6b71ec2c74dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2abc3018cace459a99c3bae89f4dffc5)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:01.056569 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db1d417-55db-4df5-8930-6b71ec2c74dd admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:01.095834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af653d-239b-4868-8948-b1568d299ed5 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:01.100253 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8df1f-99f2-4bd0-9ea8-c73cc66fcc31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:01.101156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8df1f-99f2-4bd0-9ea8-c73cc66fcc31 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:01.101544 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8df1f-99f2-4bd0-9ea8-c73cc66fcc31 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:01.101921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8df1f-99f2-4bd0-9ea8-c73cc66fcc31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:01.116013 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8df1f-99f2-4bd0-9ea8-c73cc66fcc31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:01.117730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8df1f-99f2-4bd0-9ea8-c73cc66fcc31 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:01.137061 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 902/1807] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:01.183415 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-3db1d417-55db-4df5-8930-6b71ec2c74dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2abc3018cace459a99c3bae89f4dffc5', 'request_id': 'req-3db1d417-55db-4df5-8930-6b71ec2c74dd'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:01.198430 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-3db1d417-55db-4df5-8930-6b71ec2c74dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2abc3018cace459a99c3bae89f4dffc5', 'request_id': 'req-3db1d417-55db-4df5-8930-6b71ec2c74dd'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:01.209761 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-3db1d417-55db-4df5-8930-6b71ec2c74dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2abc3018cace459a99c3bae89f4dffc5', 'request_id': 'req-3db1d417-55db-4df5-8930-6b71ec2c74dd'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:01.227037 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-3db1d417-55db-4df5-8930-6b71ec2c74dd admin admin] The token cache is being invalidate because project 2abc3018cace459a99c3bae89f4dffc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:01.227392 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-3db1d417-55db-4df5-8930-6b71ec2c74dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:01.228316 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-3db1d417-55db-4df5-8930-6b71ec2c74dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:01.230302 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 906/1808] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:00 2022] DELETE /identity/v3/projects/2abc3018cace459a99c3bae89f4dffc5 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:01.234841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8df1f-99f2-4bd0-9ea8-c73cc66fcc31 service neutron] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:01.247322 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8df1f-99f2-4bd0-9ea8-c73cc66fcc31 service neutron] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:01.358488 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc14d64-6333-4563-9b8c-588b100e27d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:01.359936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc14d64-6333-4563-9b8c-588b100e27d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:01.361488 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc14d64-6333-4563-9b8c-588b100e27d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:01.361978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc14d64-6333-4563-9b8c-588b100e27d2 admin admin] PATH_INFO: `/v3/users/40b8b03b013348a385de2fd1626e941b` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:01.363630 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc14d64-6333-4563-9b8c-588b100e27d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40b8b03b013348a385de2fd1626e941b)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:01.365793 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc14d64-6333-4563-9b8c-588b100e27d2 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:01.438846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-9fc14d64-6333-4563-9b8c-588b100e27d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40b8b03b013348a385de2fd1626e941b'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:01.448299 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-9fc14d64-6333-4563-9b8c-588b100e27d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40b8b03b013348a385de2fd1626e941b'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:01.449462 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-9fc14d64-6333-4563-9b8c-588b100e27d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40b8b03b013348a385de2fd1626e941b'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:01.467574 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-9fc14d64-6333-4563-9b8c-588b100e27d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40b8b03b013348a385de2fd1626e941b'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:01.468587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-9fc14d64-6333-4563-9b8c-588b100e27d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40b8b03b013348a385de2fd1626e941b'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:01.497203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 903/1809] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:01 2022] DELETE /identity/v3/users/40b8b03b013348a385de2fd1626e941b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:01.515248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1d417-55db-4df5-8930-6b71ec2c74dd admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:01.517603 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-3db1d417-55db-4df5-8930-6b71ec2c74dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:01.544326 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1d417-55db-4df5-8930-6b71ec2c74dd admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:01.656477 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f209cc-579c-46c7-ba2b-d999631fff32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:01.657968 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f209cc-579c-46c7-ba2b-d999631fff32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:01.658239 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f209cc-579c-46c7-ba2b-d999631fff32 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:01.658571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f209cc-579c-46c7-ba2b-d999631fff32 admin admin] PATH_INFO: `/v3/projects/2378990540cc490aa181b4201cc1b37f` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:01.660456 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f209cc-579c-46c7-ba2b-d999631fff32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2378990540cc490aa181b4201cc1b37f)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:01.663009 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f209cc-579c-46c7-ba2b-d999631fff32 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:01.728982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:01.741864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-32f209cc-579c-46c7-ba2b-d999631fff32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2378990540cc490aa181b4201cc1b37f', 'request_id': 'req-32f209cc-579c-46c7-ba2b-d999631fff32'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:01.753718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-32f209cc-579c-46c7-ba2b-d999631fff32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2378990540cc490aa181b4201cc1b37f', 'request_id': 'req-32f209cc-579c-46c7-ba2b-d999631fff32'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:01.761712 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-32f209cc-579c-46c7-ba2b-d999631fff32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2378990540cc490aa181b4201cc1b37f', 'request_id': 'req-32f209cc-579c-46c7-ba2b-d999631fff32'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:01.775251 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-32f209cc-579c-46c7-ba2b-d999631fff32 admin admin] The token cache is being invalidate because project 2378990540cc490aa181b4201cc1b37f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:01.775534 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-32f209cc-579c-46c7-ba2b-d999631fff32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:01.776740 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-32f209cc-579c-46c7-ba2b-d999631fff32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:01.778370 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 907/1810] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:01 2022] DELETE /identity/v3/projects/2378990540cc490aa181b4201cc1b37f => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:01.782846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc14d64-6333-4563-9b8c-588b100e27d2 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:01.784571 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-9fc14d64-6333-4563-9b8c-588b100e27d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:01.795944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:01.805495 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc14d64-6333-4563-9b8c-588b100e27d2 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:01.903951 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f209cc-579c-46c7-ba2b-d999631fff32 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:01.905816 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-32f209cc-579c-46c7-ba2b-d999631fff32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:01.912185 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a908f9-da28-4802-914a-c7fcf9e9170e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:01.914208 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a908f9-da28-4802-914a-c7fcf9e9170e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:01.914697 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a908f9-da28-4802-914a-c7fcf9e9170e admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:01.915292 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a908f9-da28-4802-914a-c7fcf9e9170e admin admin] PATH_INFO: `/v3/users/03c82e90e4254a79abb6be4bebd91a3d` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:01.919711 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a908f9-da28-4802-914a-c7fcf9e9170e admin admin] RBAC: Authorizing `identity:delete_user(user_id=03c82e90e4254a79abb6be4bebd91a3d)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:01.923184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a908f9-da28-4802-914a-c7fcf9e9170e admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:01.950184 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f209cc-579c-46c7-ba2b-d999631fff32 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:01.963015 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a31a6-c1d2-4682-8bab-4ed7d3345b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:01.965017 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a31a6-c1d2-4682-8bab-4ed7d3345b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:01.965640 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a31a6-c1d2-4682-8bab-4ed7d3345b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:01.966249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a31a6-c1d2-4682-8bab-4ed7d3345b1b admin admin] PATH_INFO: `/v3/projects/cef1b4ce3d1e4dd48fce791682b1f8b0` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:01.971652 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a31a6-c1d2-4682-8bab-4ed7d3345b1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cef1b4ce3d1e4dd48fce791682b1f8b0)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:01.976856 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a31a6-c1d2-4682-8bab-4ed7d3345b1b admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:02.059182 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-16a908f9-da28-4802-914a-c7fcf9e9170e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03c82e90e4254a79abb6be4bebd91a3d'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:02.079971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-16a908f9-da28-4802-914a-c7fcf9e9170e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03c82e90e4254a79abb6be4bebd91a3d'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:02.081124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-16a908f9-da28-4802-914a-c7fcf9e9170e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03c82e90e4254a79abb6be4bebd91a3d'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:02.123590 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-16a908f9-da28-4802-914a-c7fcf9e9170e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03c82e90e4254a79abb6be4bebd91a3d'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:02.125607 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-16a908f9-da28-4802-914a-c7fcf9e9170e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03c82e90e4254a79abb6be4bebd91a3d'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:02.127495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-a02a31a6-c1d2-4682-8bab-4ed7d3345b1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cef1b4ce3d1e4dd48fce791682b1f8b0', 'request_id': 'req-a02a31a6-c1d2-4682-8bab-4ed7d3345b1b'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:02.174977 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-a02a31a6-c1d2-4682-8bab-4ed7d3345b1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cef1b4ce3d1e4dd48fce791682b1f8b0', 'request_id': 'req-a02a31a6-c1d2-4682-8bab-4ed7d3345b1b'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:02.206971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 904/1811] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:01 2022] DELETE /identity/v3/users/03c82e90e4254a79abb6be4bebd91a3d => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:02.222367 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a908f9-da28-4802-914a-c7fcf9e9170e admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:02.225534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-16a908f9-da28-4802-914a-c7fcf9e9170e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:02.246877 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-a02a31a6-c1d2-4682-8bab-4ed7d3345b1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cef1b4ce3d1e4dd48fce791682b1f8b0', 'request_id': 'req-a02a31a6-c1d2-4682-8bab-4ed7d3345b1b'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:02.337063 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-a02a31a6-c1d2-4682-8bab-4ed7d3345b1b admin admin] The token cache is being invalidate because project cef1b4ce3d1e4dd48fce791682b1f8b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:02.337774 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-a02a31a6-c1d2-4682-8bab-4ed7d3345b1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:02.339062 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-a02a31a6-c1d2-4682-8bab-4ed7d3345b1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:02.345773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 908/1812] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:01 2022] DELETE /identity/v3/projects/cef1b4ce3d1e4dd48fce791682b1f8b0 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:02.356824 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70411c80-d9f2-4fcb-9482-2a174904f31a None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:02.357291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70411c80-d9f2-4fcb-9482-2a174904f31a None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:02.357751 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70411c80-d9f2-4fcb-9482-2a174904f31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:02.396310 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-70411c80-d9f2-4fcb-9482-2a174904f31a None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:02.407552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-70411c80-d9f2-4fcb-9482-2a174904f31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:02.446518 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a908f9-da28-4802-914a-c7fcf9e9170e admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:02.493158 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a75bfe-5fd2-48a9-b98a-9914a0bffd69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:02.493158 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a75bfe-5fd2-48a9-b98a-9914a0bffd69 service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:02.494281 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a75bfe-5fd2-48a9-b98a-9914a0bffd69 service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:02.495410 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a75bfe-5fd2-48a9-b98a-9914a0bffd69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:02.500312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-59a75bfe-5fd2-48a9-b98a-9914a0bffd69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:02.573556 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:02.600260 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 909/1813] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:02.605927 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e67ad-b717-48e9-b39b-8f10058bb703 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:02.607625 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e67ad-b717-48e9-b39b-8f10058bb703 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:02.608401 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e67ad-b717-48e9-b39b-8f10058bb703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:02.683978 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-b66e67ad-b717-48e9-b39b-8f10058bb703 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:02.687625 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-b66e67ad-b717-48e9-b39b-8f10058bb703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:02.695063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a75bfe-5fd2-48a9-b98a-9914a0bffd69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:02.696727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a75bfe-5fd2-48a9-b98a-9914a0bffd69 service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:02.720023 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 910/1814] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:02.724429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 905/1815] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:02 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:02.725167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e67ad-b717-48e9-b39b-8f10058bb703 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:02.754248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e67ad-b717-48e9-b39b-8f10058bb703 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:02.764823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a39f91-bc6f-4083-9c47-c724377eb084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:02.766352 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a39f91-bc6f-4083-9c47-c724377eb084 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:02.767082 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a39f91-bc6f-4083-9c47-c724377eb084 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:02.767805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a39f91-bc6f-4083-9c47-c724377eb084 admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:02.769272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a39f91-bc6f-4083-9c47-c724377eb084 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:02.769817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a39f91-bc6f-4083-9c47-c724377eb084 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:02.775148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a39f91-bc6f-4083-9c47-c724377eb084 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:02.811755 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 911/1816] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:02.816212 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a75bfe-5fd2-48a9-b98a-9914a0bffd69 service cinder] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:02.855260 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a75bfe-5fd2-48a9-b98a-9914a0bffd69 service cinder] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:02.868313 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ded2ce-2831-458b-bb95-a06d40e09154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:02.870524 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ded2ce-2831-458b-bb95-a06d40e09154 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:02.871129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ded2ce-2831-458b-bb95-a06d40e09154 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:02.871489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ded2ce-2831-458b-bb95-a06d40e09154 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:02.873137 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ded2ce-2831-458b-bb95-a06d40e09154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:02.878219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ded2ce-2831-458b-bb95-a06d40e09154 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:02.958750 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 906/1817] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:02 2022] POST /identity/v3/projects => generated 401 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:02.963370 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a39f91-bc6f-4083-9c47-c724377eb084 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:02.981161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a39f91-bc6f-4083-9c47-c724377eb084 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:03.093893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:03.157619 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab524b-6711-4a00-800b-659ecc177626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:03.158318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab524b-6711-4a00-800b-659ecc177626 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:03.158607 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab524b-6711-4a00-800b-659ecc177626 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:03.158886 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab524b-6711-4a00-800b-659ecc177626 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:03.160007 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab524b-6711-4a00-800b-659ecc177626 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:03.167469 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab524b-6711-4a00-800b-659ecc177626 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:03.212514 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 912/1818] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:02 2022] POST /identity/v3/users => generated 301 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:40:03.215277 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ded2ce-2831-458b-bb95-a06d40e09154 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:03.244062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ded2ce-2831-458b-bb95-a06d40e09154 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:03.254008 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02b632-417a-42d3-b2b0-64a70c9268b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:03.255870 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02b632-417a-42d3-b2b0-64a70c9268b3 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:03.257120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02b632-417a-42d3-b2b0-64a70c9268b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:03.257120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02b632-417a-42d3-b2b0-64a70c9268b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:03.258386 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02b632-417a-42d3-b2b0-64a70c9268b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:03.260292 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02b632-417a-42d3-b2b0-64a70c9268b3 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:03.275115 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 907/1819] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:03.279499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab524b-6711-4a00-800b-659ecc177626 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:03.294443 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab524b-6711-4a00-800b-659ecc177626 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:03.301310 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8fd8d-a125-4095-9f04-af772a21c1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:03.302120 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8fd8d-a125-4095-9f04-af772a21c1be admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:03.302487 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8fd8d-a125-4095-9f04-af772a21c1be admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:03.302844 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8fd8d-a125-4095-9f04-af772a21c1be admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:03.303929 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8fd8d-a125-4095-9f04-af772a21c1be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:03.306648 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8fd8d-a125-4095-9f04-af772a21c1be admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:03.320986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 913/1820] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:03.325448 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02b632-417a-42d3-b2b0-64a70c9268b3 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:03.339695 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02b632-417a-42d3-b2b0-64a70c9268b3 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:03.349601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d92f4a-8b33-4245-8015-16b9739e249f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:03.351252 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d92f4a-8b33-4245-8015-16b9739e249f admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:03.351252 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d92f4a-8b33-4245-8015-16b9739e249f admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:03.351615 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d92f4a-8b33-4245-8015-16b9739e249f admin admin] PATH_INFO: `/v3/projects/01ca03bf6ba14cceb8d90f0255d9db85/users/ed950511eb4247c0907f4654ef0c5a6e/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:03.395094 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d92f4a-8b33-4245-8015-16b9739e249f admin admin] RBAC: Authorizing `identity:create_grant(project_id=01ca03bf6ba14cceb8d90f0255d9db85, user_id=ed950511eb4247c0907f4654ef0c5a6e, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:03.408734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d92f4a-8b33-4245-8015-16b9739e249f admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:03.449374 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 908/1821] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:03 2022] PUT /identity/v3/projects/01ca03bf6ba14cceb8d90f0255d9db85/users/ed950511eb4247c0907f4654ef0c5a6e/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:03.451932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34c73a-0e62-47f4-868e-37fd47597901 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:03.452226 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34c73a-0e62-47f4-868e-37fd47597901 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:03.452535 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34c73a-0e62-47f4-868e-37fd47597901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:03.518059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-6b34c73a-0e62-47f4-868e-37fd47597901 None None] MFA Rules not processed for user `ed950511eb4247c0907f4654ef0c5a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:03.522957 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-6b34c73a-0e62-47f4-868e-37fd47597901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:03.525485 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d92f4a-8b33-4245-8015-16b9739e249f admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:03.528317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-47d92f4a-8b33-4245-8015-16b9739e249f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:03.685153 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 914/1822] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:03.689885 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34c73a-0e62-47f4-868e-37fd47597901 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:03.723911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d92f4a-8b33-4245-8015-16b9739e249f admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:03.734337 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34c73a-0e62-47f4-868e-37fd47597901 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:03.735811 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632437e8-ae1d-40c0-9cb7-d053ecd5d3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:03.736822 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632437e8-ae1d-40c0-9cb7-d053ecd5d3a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:03.737223 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632437e8-ae1d-40c0-9cb7-d053ecd5d3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:03.737786 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632437e8-ae1d-40c0-9cb7-d053ecd5d3a9 admin admin] PATH_INFO: `/v3/projects/d00bcd228fac40b49c0b17661b9f601b` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:03.740389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632437e8-ae1d-40c0-9cb7-d053ecd5d3a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d00bcd228fac40b49c0b17661b9f601b)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:03.745368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632437e8-ae1d-40c0-9cb7-d053ecd5d3a9 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:03.747519 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c554ae-33dc-49f0-9848-cf53e87b8836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:03.749621 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c554ae-33dc-49f0-9848-cf53e87b8836 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:03.750268 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c554ae-33dc-49f0-9848-cf53e87b8836 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:03.750754 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c554ae-33dc-49f0-9848-cf53e87b8836 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:03.751727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c554ae-33dc-49f0-9848-cf53e87b8836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:03.755503 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c554ae-33dc-49f0-9848-cf53e87b8836 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:03.798959 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 915/1823] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:03 2022] POST /identity/v3/projects => generated 399 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:03.806815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c554ae-33dc-49f0-9848-cf53e87b8836 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:03.874917 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-632437e8-ae1d-40c0-9cb7-d053ecd5d3a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd00bcd228fac40b49c0b17661b9f601b', 'request_id': 'req-632437e8-ae1d-40c0-9cb7-d053ecd5d3a9'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:03.888966 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-632437e8-ae1d-40c0-9cb7-d053ecd5d3a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd00bcd228fac40b49c0b17661b9f601b', 'request_id': 'req-632437e8-ae1d-40c0-9cb7-d053ecd5d3a9'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:03.897280 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-632437e8-ae1d-40c0-9cb7-d053ecd5d3a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd00bcd228fac40b49c0b17661b9f601b', 'request_id': 'req-632437e8-ae1d-40c0-9cb7-d053ecd5d3a9'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:03.916715 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c554ae-33dc-49f0-9848-cf53e87b8836 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:03.917002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-632437e8-ae1d-40c0-9cb7-d053ecd5d3a9 admin admin] The token cache is being invalidate because project d00bcd228fac40b49c0b17661b9f601b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:03.917285 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-632437e8-ae1d-40c0-9cb7-d053ecd5d3a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:03.918059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-632437e8-ae1d-40c0-9cb7-d053ecd5d3a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:03.920503 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 909/1824] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:03 2022] DELETE /identity/v3/projects/d00bcd228fac40b49c0b17661b9f601b => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:03.925820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c6313-5627-4edc-af40-8198cf31b99f None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:03.926110 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c6313-5627-4edc-af40-8198cf31b99f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:03.926396 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c6313-5627-4edc-af40-8198cf31b99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:03.939379 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:03.947920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:03.960143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-764c6313-5627-4edc-af40-8198cf31b99f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:03.962302 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-764c6313-5627-4edc-af40-8198cf31b99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:04.003502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86206afb-179f-47cb-836d-63a52a15c12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.004834 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86206afb-179f-47cb-836d-63a52a15c12c admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.005345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86206afb-179f-47cb-836d-63a52a15c12c admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.005817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86206afb-179f-47cb-836d-63a52a15c12c admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.007062 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86206afb-179f-47cb-836d-63a52a15c12c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.010361 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86206afb-179f-47cb-836d-63a52a15c12c admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.048872 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 916/1825] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:03 2022] POST /identity/v3/users => generated 306 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:04.054492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86206afb-179f-47cb-836d-63a52a15c12c admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.057012 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-86206afb-179f-47cb-836d-63a52a15c12c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:04.073160 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86206afb-179f-47cb-836d-63a52a15c12c admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.079003 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f877dfe-0f12-4567-ad25-ffaa29b5cfc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.079696 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f877dfe-0f12-4567-ad25-ffaa29b5cfc6 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.079985 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f877dfe-0f12-4567-ad25-ffaa29b5cfc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.080277 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f877dfe-0f12-4567-ad25-ffaa29b5cfc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.081016 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f877dfe-0f12-4567-ad25-ffaa29b5cfc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.081248 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 910/1826] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:04.082409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c6313-5627-4edc-af40-8198cf31b99f None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.083950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-764c6313-5627-4edc-af40-8198cf31b99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:04.090139 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f877dfe-0f12-4567-ad25-ffaa29b5cfc6 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.099266 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c6313-5627-4edc-af40-8198cf31b99f None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.101965 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 917/1827] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:04.104735 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b072bab-f0e5-41a7-b08c-a44d9f9098bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.105973 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b072bab-f0e5-41a7-b08c-a44d9f9098bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.106253 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b072bab-f0e5-41a7-b08c-a44d9f9098bf admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.106579 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b072bab-f0e5-41a7-b08c-a44d9f9098bf admin admin] PATH_INFO: `/v3/users/226f6f56564b42e8be142a727efc37e6` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.106730 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401bb85-0f68-4406-b2fa-f99c82512689 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.106959 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401bb85-0f68-4406-b2fa-f99c82512689 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.107245 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401bb85-0f68-4406-b2fa-f99c82512689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.107921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b072bab-f0e5-41a7-b08c-a44d9f9098bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=226f6f56564b42e8be142a727efc37e6)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.109918 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b072bab-f0e5-41a7-b08c-a44d9f9098bf admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.133129 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-f401bb85-0f68-4406-b2fa-f99c82512689 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:04.134776 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-f401bb85-0f68-4406-b2fa-f99c82512689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:04.150581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 918/1828] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:04.154699 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401bb85-0f68-4406-b2fa-f99c82512689 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.172511 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401bb85-0f68-4406-b2fa-f99c82512689 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.183533 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0717ad-5aae-4d23-87a9-88498982f457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.185116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0717ad-5aae-4d23-87a9-88498982f457 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.185677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0717ad-5aae-4d23-87a9-88498982f457 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.186181 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0717ad-5aae-4d23-87a9-88498982f457 admin admin] PATH_INFO: `/v3/projects/86547ced096442f6bec878a65e3efc98/users/3e5b45f9f2334e209f2acbe8d056a7f2/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.202477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3b072bab-f0e5-41a7-b08c-a44d9f9098bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '226f6f56564b42e8be142a727efc37e6'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:04.214356 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0717ad-5aae-4d23-87a9-88498982f457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86547ced096442f6bec878a65e3efc98, user_id=3e5b45f9f2334e209f2acbe8d056a7f2, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.216813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3b072bab-f0e5-41a7-b08c-a44d9f9098bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '226f6f56564b42e8be142a727efc37e6'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:04.218067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3b072bab-f0e5-41a7-b08c-a44d9f9098bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '226f6f56564b42e8be142a727efc37e6'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:04.218209 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0717ad-5aae-4d23-87a9-88498982f457 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.235518 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3b072bab-f0e5-41a7-b08c-a44d9f9098bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '226f6f56564b42e8be142a727efc37e6'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:04.236770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3b072bab-f0e5-41a7-b08c-a44d9f9098bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '226f6f56564b42e8be142a727efc37e6'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:04.237337 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 919/1829] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:04 2022] PUT /identity/v3/projects/86547ced096442f6bec878a65e3efc98/users/3e5b45f9f2334e209f2acbe8d056a7f2/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:04.238056 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0717ad-5aae-4d23-87a9-88498982f457 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.245289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-3a0717ad-5aae-4d23-87a9-88498982f457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:04.270120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 911/1830] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:04 2022] DELETE /identity/v3/users/226f6f56564b42e8be142a727efc37e6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:04.270881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b072bab-f0e5-41a7-b08c-a44d9f9098bf admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.277413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3b072bab-f0e5-41a7-b08c-a44d9f9098bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:04.367305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0717ad-5aae-4d23-87a9-88498982f457 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.377451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233a5c4-48bc-4355-ba91-7c6fb33749b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.378145 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233a5c4-48bc-4355-ba91-7c6fb33749b5 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.379082 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233a5c4-48bc-4355-ba91-7c6fb33749b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.379362 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233a5c4-48bc-4355-ba91-7c6fb33749b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.380419 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233a5c4-48bc-4355-ba91-7c6fb33749b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:04.380742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233a5c4-48bc-4355-ba91-7c6fb33749b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.382390 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233a5c4-48bc-4355-ba91-7c6fb33749b5 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.382792 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b072bab-f0e5-41a7-b08c-a44d9f9098bf admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.388261 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1c038-d7f7-4f0f-aadb-3824fa5f5b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.389165 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1c038-d7f7-4f0f-aadb-3824fa5f5b4c admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.389512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1c038-d7f7-4f0f-aadb-3824fa5f5b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.389862 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1c038-d7f7-4f0f-aadb-3824fa5f5b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.390697 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1c038-d7f7-4f0f-aadb-3824fa5f5b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.392485 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1c038-d7f7-4f0f-aadb-3824fa5f5b4c admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.404918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 920/1831] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:04.405145 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 912/1832] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:04.406695 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c659d7-d4d4-4e68-a797-b5d8c89d4e89 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.407003 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c659d7-d4d4-4e68-a797-b5d8c89d4e89 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.407285 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c659d7-d4d4-4e68-a797-b5d8c89d4e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.410390 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1c038-d7f7-4f0f-aadb-3824fa5f5b4c admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.423532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1c038-d7f7-4f0f-aadb-3824fa5f5b4c admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.428341 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-59c659d7-d4d4-4e68-a797-b5d8c89d4e89 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:04.428557 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44be06-aab3-4391-88c8-8b24ab3966be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.429256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44be06-aab3-4391-88c8-8b24ab3966be admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.429527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44be06-aab3-4391-88c8-8b24ab3966be admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.429679 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-59c659d7-d4d4-4e68-a797-b5d8c89d4e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:04.429816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44be06-aab3-4391-88c8-8b24ab3966be admin admin] PATH_INFO: `/v3/projects/86547ced096442f6bec878a65e3efc98/users/3e5b45f9f2334e209f2acbe8d056a7f2/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.433685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44be06-aab3-4391-88c8-8b24ab3966be admin admin] RBAC: Authorizing `identity:create_grant(project_id=86547ced096442f6bec878a65e3efc98, user_id=3e5b45f9f2334e209f2acbe8d056a7f2, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.436399 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44be06-aab3-4391-88c8-8b24ab3966be admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.441794 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 921/1833] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:04.441794 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c659d7-d4d4-4e68-a797-b5d8c89d4e89 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.444590 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:04.455446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 913/1834] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:04 2022] PUT /identity/v3/projects/86547ced096442f6bec878a65e3efc98/users/3e5b45f9f2334e209f2acbe8d056a7f2/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:04.455446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44be06-aab3-4391-88c8-8b24ab3966be admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.462093 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c659d7-d4d4-4e68-a797-b5d8c89d4e89 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.463157 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-ff44be06-aab3-4391-88c8-8b24ab3966be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:04.549511 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44be06-aab3-4391-88c8-8b24ab3966be admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.552950 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58ac04-ce76-4a4e-b8bd-66a34b17d280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.553133 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac33d9-d3c6-46a4-8b08-75e8c34d16cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.553768 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac33d9-d3c6-46a4-8b08-75e8c34d16cb service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.554052 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac33d9-d3c6-46a4-8b08-75e8c34d16cb service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.554333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac33d9-d3c6-46a4-8b08-75e8c34d16cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.554423 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58ac04-ce76-4a4e-b8bd-66a34b17d280 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.554915 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58ac04-ce76-4a4e-b8bd-66a34b17d280 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.555437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58ac04-ce76-4a4e-b8bd-66a34b17d280 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.556948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58ac04-ce76-4a4e-b8bd-66a34b17d280 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.561451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58ac04-ce76-4a4e-b8bd-66a34b17d280 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.564540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac33d9-d3c6-46a4-8b08-75e8c34d16cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.566034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac33d9-d3c6-46a4-8b08-75e8c34d16cb service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.567711 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:04.577768 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 914/1835] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:04.579333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660f8f2-43a8-42d3-ac42-1d4cc5b85506 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.579612 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660f8f2-43a8-42d3-ac42-1d4cc5b85506 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.579981 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660f8f2-43a8-42d3-ac42-1d4cc5b85506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.610960 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-9660f8f2-43a8-42d3-ac42-1d4cc5b85506 None None] MFA Rules not processed for user `3e5b45f9f2334e209f2acbe8d056a7f2`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:04.612199 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-9660f8f2-43a8-42d3-ac42-1d4cc5b85506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:04.616040 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 922/1836] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:04 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:04.620937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58ac04-ce76-4a4e-b8bd-66a34b17d280 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.718788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 915/1837] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:04 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:04.724894 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58ac04-ce76-4a4e-b8bd-66a34b17d280 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.730008 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2293a68-b210-4fd9-be0b-779ffda24825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.730683 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2293a68-b210-4fd9-be0b-779ffda24825 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.730960 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2293a68-b210-4fd9-be0b-779ffda24825 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.731237 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2293a68-b210-4fd9-be0b-779ffda24825 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.732047 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2293a68-b210-4fd9-be0b-779ffda24825 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.733249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660f8f2-43a8-42d3-ac42-1d4cc5b85506 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.734062 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2293a68-b210-4fd9-be0b-779ffda24825 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.734933 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-9660f8f2-43a8-42d3-ac42-1d4cc5b85506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:04.760430 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 923/1838] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:04 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:40:04.761669 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2293a68-b210-4fd9-be0b-779ffda24825 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.773269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2293a68-b210-4fd9-be0b-779ffda24825 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.779616 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae7aea-4494-4721-94a5-0e7d4f8347b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.780172 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae7aea-4494-4721-94a5-0e7d4f8347b3 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.780764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae7aea-4494-4721-94a5-0e7d4f8347b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.780918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae7aea-4494-4721-94a5-0e7d4f8347b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.781692 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae7aea-4494-4721-94a5-0e7d4f8347b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.783575 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae7aea-4494-4721-94a5-0e7d4f8347b3 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.801341 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 924/1839] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:04.802652 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae7aea-4494-4721-94a5-0e7d4f8347b3 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.818521 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae7aea-4494-4721-94a5-0e7d4f8347b3 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.821129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660f8f2-43a8-42d3-ac42-1d4cc5b85506 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.824744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263bb6f1-ded2-4baf-a64e-317c6e4820ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.825327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ab906-4198-4309-9d10-26589fe5cca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.825460 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bb6f1-ded2-4baf-a64e-317c6e4820ec service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.825727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bb6f1-ded2-4baf-a64e-317c6e4820ec service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.825997 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bb6f1-ded2-4baf-a64e-317c6e4820ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.826144 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ab906-4198-4309-9d10-26589fe5cca0 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.826591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ab906-4198-4309-9d10-26589fe5cca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.826923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ab906-4198-4309-9d10-26589fe5cca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.827687 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ab906-4198-4309-9d10-26589fe5cca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.830501 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ab906-4198-4309-9d10-26589fe5cca0 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.836840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263bb6f1-ded2-4baf-a64e-317c6e4820ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.838867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263bb6f1-ded2-4baf-a64e-317c6e4820ec service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.846706 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 925/1840] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:04.850457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ab906-4198-4309-9d10-26589fe5cca0 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.856833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 916/1841] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:04 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:40:04.870825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ab906-4198-4309-9d10-26589fe5cca0 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.877745 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796259f8-73b4-429f-b137-7bd62104b568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.879137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796259f8-73b4-429f-b137-7bd62104b568 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.879552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796259f8-73b4-429f-b137-7bd62104b568 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.880286 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796259f8-73b4-429f-b137-7bd62104b568 admin admin] PATH_INFO: `/v3/projects/0fd5f130eda146eba0585d6d348b204b/users/be4292b5f7c5414997cb1277968a8bc6/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.914251 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796259f8-73b4-429f-b137-7bd62104b568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd5f130eda146eba0585d6d348b204b, user_id=be4292b5f7c5414997cb1277968a8bc6, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.919611 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796259f8-73b4-429f-b137-7bd62104b568 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.949178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 926/1842] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:04 2022] PUT /identity/v3/projects/0fd5f130eda146eba0585d6d348b204b/users/be4292b5f7c5414997cb1277968a8bc6/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:04.966202 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada06a83-bc30-4a8d-9e0e-9a64f91c802e None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.966721 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada06a83-bc30-4a8d-9e0e-9a64f91c802e None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.967298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada06a83-bc30-4a8d-9e0e-9a64f91c802e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:05.004953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796259f8-73b4-429f-b137-7bd62104b568 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:05.021437 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-ada06a83-bc30-4a8d-9e0e-9a64f91c802e None None] MFA Rules not processed for user `be4292b5f7c5414997cb1277968a8bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:05.023397 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-ada06a83-bc30-4a8d-9e0e-9a64f91c802e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:05.062080 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:05.134955 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796259f8-73b4-429f-b137-7bd62104b568 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:05.142276 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad6cc1-a575-4b0f-8732-c711e31b921c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:05.143434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad6cc1-a575-4b0f-8732-c711e31b921c service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:05.143972 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad6cc1-a575-4b0f-8732-c711e31b921c service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:05.144912 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad6cc1-a575-4b0f-8732-c711e31b921c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:05.146607 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-39ad6cc1-a575-4b0f-8732-c711e31b921c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:05.165264 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 917/1843] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:05.174198 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada06a83-bc30-4a8d-9e0e-9a64f91c802e None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:05.244119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:05.265761 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada06a83-bc30-4a8d-9e0e-9a64f91c802e None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:05.271562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c2f87-277c-48be-a3e5-52812324dfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:05.277739 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c2f87-277c-48be-a3e5-52812324dfd2 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:05.278841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c2f87-277c-48be-a3e5-52812324dfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:05.279098 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad6cc1-a575-4b0f-8732-c711e31b921c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:05.279886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c2f87-277c-48be-a3e5-52812324dfd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:05.280525 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad6cc1-a575-4b0f-8732-c711e31b921c service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:05.281906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c2f87-277c-48be-a3e5-52812324dfd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:05.286773 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c2f87-277c-48be-a3e5-52812324dfd2 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:05.294033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 927/1844] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:05 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:05.295053 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad6cc1-a575-4b0f-8732-c711e31b921c service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:05.296908 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-39ad6cc1-a575-4b0f-8732-c711e31b921c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:05.313659 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad6cc1-a575-4b0f-8732-c711e31b921c service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:05.340345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17026651-9265-44f4-a884-1909a5c8235f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:05.341541 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17026651-9265-44f4-a884-1909a5c8235f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:05.341927 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17026651-9265-44f4-a884-1909a5c8235f admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:05.342303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17026651-9265-44f4-a884-1909a5c8235f admin admin] PATH_INFO: `/v3/projects/2b66f19119de450b9e7fee99b7918e30` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:05.343766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 918/1845] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:05 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:40:05.344091 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17026651-9265-44f4-a884-1909a5c8235f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b66f19119de450b9e7fee99b7918e30)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:05.348467 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17026651-9265-44f4-a884-1909a5c8235f admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:05.350338 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c2f87-277c-48be-a3e5-52812324dfd2 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:05.464589 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c2f87-277c-48be-a3e5-52812324dfd2 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:05.472228 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0405767-d03c-4ad8-a6ee-acbbb441098b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:05.474119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0405767-d03c-4ad8-a6ee-acbbb441098b admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:05.474776 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0405767-d03c-4ad8-a6ee-acbbb441098b admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:05.475074 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0405767-d03c-4ad8-a6ee-acbbb441098b admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:05.475804 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0405767-d03c-4ad8-a6ee-acbbb441098b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:05.479516 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0405767-d03c-4ad8-a6ee-acbbb441098b admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:05.496253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-17026651-9265-44f4-a884-1909a5c8235f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b66f19119de450b9e7fee99b7918e30', 'request_id': 'req-17026651-9265-44f4-a884-1909a5c8235f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:05.509263 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-17026651-9265-44f4-a884-1909a5c8235f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b66f19119de450b9e7fee99b7918e30', 'request_id': 'req-17026651-9265-44f4-a884-1909a5c8235f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:05.523398 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-17026651-9265-44f4-a884-1909a5c8235f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b66f19119de450b9e7fee99b7918e30', 'request_id': 'req-17026651-9265-44f4-a884-1909a5c8235f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:05.528589 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 919/1846] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:05 2022] POST /identity/v3/users => generated 300 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:40:05.537550 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0405767-d03c-4ad8-a6ee-acbbb441098b admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:05.546821 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-17026651-9265-44f4-a884-1909a5c8235f admin admin] The token cache is being invalidate because project 2b66f19119de450b9e7fee99b7918e30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:05.547099 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-17026651-9265-44f4-a884-1909a5c8235f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:05.547900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-17026651-9265-44f4-a884-1909a5c8235f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:05.550685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 928/1847] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:05 2022] DELETE /identity/v3/projects/2b66f19119de450b9e7fee99b7918e30 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:05.555285 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17026651-9265-44f4-a884-1909a5c8235f admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:05.558305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-17026651-9265-44f4-a884-1909a5c8235f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:05.559704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0405767-d03c-4ad8-a6ee-acbbb441098b admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:05.658707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:05.672411 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17026651-9265-44f4-a884-1909a5c8235f admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:05.679411 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e2f9f-13b5-4224-b9ba-d07dccf8f458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:05.681026 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e2f9f-13b5-4224-b9ba-d07dccf8f458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:05.681418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e2f9f-13b5-4224-b9ba-d07dccf8f458 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:05.681766 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e2f9f-13b5-4224-b9ba-d07dccf8f458 admin admin] PATH_INFO: `/v3/users/ee2c89a61713457c9e3febf7ad85a1a9` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:05.683910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e2f9f-13b5-4224-b9ba-d07dccf8f458 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee2c89a61713457c9e3febf7ad85a1a9)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:05.686627 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e2f9f-13b5-4224-b9ba-d07dccf8f458 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:05.739232 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b701c79-ac29-49a4-b6c1-af8f6376adff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:05.740600 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b701c79-ac29-49a4-b6c1-af8f6376adff admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:05.740958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b701c79-ac29-49a4-b6c1-af8f6376adff admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:05.741347 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b701c79-ac29-49a4-b6c1-af8f6376adff admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:05.742211 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b701c79-ac29-49a4-b6c1-af8f6376adff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:05.748856 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b701c79-ac29-49a4-b6c1-af8f6376adff admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:05.764087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 920/1848] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:05.770877 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b701c79-ac29-49a4-b6c1-af8f6376adff admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:05.773766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-0b701c79-ac29-49a4-b6c1-af8f6376adff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:05.788629 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-5f8e2f9f-13b5-4224-b9ba-d07dccf8f458 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee2c89a61713457c9e3febf7ad85a1a9'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:05.799052 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b701c79-ac29-49a4-b6c1-af8f6376adff admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:05.808749 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-5f8e2f9f-13b5-4224-b9ba-d07dccf8f458 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee2c89a61713457c9e3febf7ad85a1a9'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:05.809892 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-5f8e2f9f-13b5-4224-b9ba-d07dccf8f458 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee2c89a61713457c9e3febf7ad85a1a9'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:05.815487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5dfb6-0901-450e-ab74-ce2e4e4e367a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:05.817059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5dfb6-0901-450e-ab74-ce2e4e4e367a admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:05.817578 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5dfb6-0901-450e-ab74-ce2e4e4e367a admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:05.818476 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5dfb6-0901-450e-ab74-ce2e4e4e367a admin admin] PATH_INFO: `/v3/projects/745051242e914579b3e15cc48a5d3a90/users/14cf7ccd9f1c4a25a123c9d329681818/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:05.841281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:05.856753 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-5f8e2f9f-13b5-4224-b9ba-d07dccf8f458 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee2c89a61713457c9e3febf7ad85a1a9'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:05.878181 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5dfb6-0901-450e-ab74-ce2e4e4e367a admin admin] RBAC: Authorizing `identity:create_grant(project_id=745051242e914579b3e15cc48a5d3a90, user_id=14cf7ccd9f1c4a25a123c9d329681818, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:05.884914 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5dfb6-0901-450e-ab74-ce2e4e4e367a admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:05.894914 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-5f8e2f9f-13b5-4224-b9ba-d07dccf8f458 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee2c89a61713457c9e3febf7ad85a1a9'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:05.916293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 929/1849] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:05 2022] DELETE /identity/v3/users/ee2c89a61713457c9e3febf7ad85a1a9 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:05.918147 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 921/1850] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:05 2022] PUT /identity/v3/projects/745051242e914579b3e15cc48a5d3a90/users/14cf7ccd9f1c4a25a123c9d329681818/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:05.926320 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e2f9f-13b5-4224-b9ba-d07dccf8f458 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:05.928870 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-5f8e2f9f-13b5-4224-b9ba-d07dccf8f458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:06.074452 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e2f9f-13b5-4224-b9ba-d07dccf8f458 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:06.082664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97cc50-14e5-4c10-8cfb-356295606047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:06.083964 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97cc50-14e5-4c10-8cfb-356295606047 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:06.084130 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97cc50-14e5-4c10-8cfb-356295606047 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:06.084960 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97cc50-14e5-4c10-8cfb-356295606047 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:06.086383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97cc50-14e5-4c10-8cfb-356295606047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:06.088673 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97cc50-14e5-4c10-8cfb-356295606047 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:06.108255 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 930/1851] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:06.110993 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5dfb6-0901-450e-ab74-ce2e4e4e367a admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:06.233886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5dfb6-0901-450e-ab74-ce2e4e4e367a admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:06.242057 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109aa1eb-87fb-4bad-b044-501f9ca64a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:06.243648 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109aa1eb-87fb-4bad-b044-501f9ca64a4a admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:06.244140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109aa1eb-87fb-4bad-b044-501f9ca64a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:06.244679 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109aa1eb-87fb-4bad-b044-501f9ca64a4a admin admin] PATH_INFO: `/v3/projects/745051242e914579b3e15cc48a5d3a90/users/14cf7ccd9f1c4a25a123c9d329681818/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:06.248400 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109aa1eb-87fb-4bad-b044-501f9ca64a4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=745051242e914579b3e15cc48a5d3a90, user_id=14cf7ccd9f1c4a25a123c9d329681818, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:06.253981 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109aa1eb-87fb-4bad-b044-501f9ca64a4a admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:06.273500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 922/1852] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:06 2022] PUT /identity/v3/projects/745051242e914579b3e15cc48a5d3a90/users/14cf7ccd9f1c4a25a123c9d329681818/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:06.292495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f08685-b800-4cb6-9c05-d6d8dcbe5766 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:06.292814 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f08685-b800-4cb6-9c05-d6d8dcbe5766 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:06.293179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f08685-b800-4cb6-9c05-d6d8dcbe5766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:06.374477 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-93f08685-b800-4cb6-9c05-d6d8dcbe5766 None None] MFA Rules not processed for user `14cf7ccd9f1c4a25a123c9d329681818`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:06.376014 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-93f08685-b800-4cb6-9c05-d6d8dcbe5766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:06.566010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109aa1eb-87fb-4bad-b044-501f9ca64a4a admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:06.570713 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-109aa1eb-87fb-4bad-b044-501f9ca64a4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:06.658418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 931/1853] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:06.684997 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f08685-b800-4cb6-9c05-d6d8dcbe5766 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:06.690068 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-93f08685-b800-4cb6-9c05-d6d8dcbe5766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:06.897726 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109aa1eb-87fb-4bad-b044-501f9ca64a4a admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:06.909237 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f08685-b800-4cb6-9c05-d6d8dcbe5766 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:06.913022 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f6961-74b2-4b01-b29b-706ac0beea30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:06.915642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f6961-74b2-4b01-b29b-706ac0beea30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:06.916085 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f6961-74b2-4b01-b29b-706ac0beea30 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:06.916524 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f6961-74b2-4b01-b29b-706ac0beea30 admin admin] PATH_INFO: `/v3/projects/f03c2c0a06b64001b77bcef997e15a6e` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:06.918983 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f6961-74b2-4b01-b29b-706ac0beea30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f03c2c0a06b64001b77bcef997e15a6e)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:06.927807 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdbccf-bf46-4953-96db-11afa5b7a4e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:06.928677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f6961-74b2-4b01-b29b-706ac0beea30 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:06.928987 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdbccf-bf46-4953-96db-11afa5b7a4e5 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:06.929563 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdbccf-bf46-4953-96db-11afa5b7a4e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:06.930868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdbccf-bf46-4953-96db-11afa5b7a4e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:06.974768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdbccf-bf46-4953-96db-11afa5b7a4e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:06.977653 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdbccf-bf46-4953-96db-11afa5b7a4e5 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:07.032010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 932/1854] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:07.106910 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:07.120623 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-b57f6961-74b2-4b01-b29b-706ac0beea30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f03c2c0a06b64001b77bcef997e15a6e', 'request_id': 'req-b57f6961-74b2-4b01-b29b-706ac0beea30'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:07.158079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-b57f6961-74b2-4b01-b29b-706ac0beea30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f03c2c0a06b64001b77bcef997e15a6e', 'request_id': 'req-b57f6961-74b2-4b01-b29b-706ac0beea30'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:07.171855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-b57f6961-74b2-4b01-b29b-706ac0beea30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f03c2c0a06b64001b77bcef997e15a6e', 'request_id': 'req-b57f6961-74b2-4b01-b29b-706ac0beea30'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:07.207861 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-b57f6961-74b2-4b01-b29b-706ac0beea30 admin admin] The token cache is being invalidate because project f03c2c0a06b64001b77bcef997e15a6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:07.210603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-b57f6961-74b2-4b01-b29b-706ac0beea30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:07.212354 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-b57f6961-74b2-4b01-b29b-706ac0beea30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:07.217875 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 923/1855] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:06 2022] DELETE /identity/v3/projects/f03c2c0a06b64001b77bcef997e15a6e => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:07.225806 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343fc82-c224-47fb-81f0-5a5cd2a8f7e3 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:07.226276 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343fc82-c224-47fb-81f0-5a5cd2a8f7e3 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:07.227219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343fc82-c224-47fb-81f0-5a5cd2a8f7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:07.283813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-1343fc82-c224-47fb-81f0-5a5cd2a8f7e3 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:07.287064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-1343fc82-c224-47fb-81f0-5a5cd2a8f7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:07.301292 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:07.320977 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c100a1-2c3c-427d-925e-12666ae5d3dc None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:07.321420 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c100a1-2c3c-427d-925e-12666ae5d3dc None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:07.321931 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c100a1-2c3c-427d-925e-12666ae5d3dc None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:07.325478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 924/1856] 10.222.0.71 () {54 vars in 921 bytes} [Thu Jul 21 20:40:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:07.361918 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5918add-0703-4f17-afbf-e98f353b781c None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:07.362463 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5918add-0703-4f17-afbf-e98f353b781c None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:07.362941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5918add-0703-4f17-afbf-e98f353b781c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:07.411472 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-c5918add-0703-4f17-afbf-e98f353b781c None None] MFA Rules not processed for user `6fbc5240f5214aaea4418052b94739e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:07.414126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-c5918add-0703-4f17-afbf-e98f353b781c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:07.549158 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 933/1857] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:07.559806 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f4fb8-ce71-4402-ba68-c64e768ab02d None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:07.560430 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f4fb8-ce71-4402-ba68-c64e768ab02d None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:07.561044 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f4fb8-ce71-4402-ba68-c64e768ab02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:07.602051 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-025f4fb8-ce71-4402-ba68-c64e768ab02d None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:07.606285 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-025f4fb8-ce71-4402-ba68-c64e768ab02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:07.624610 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 934/1858] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:07.629811 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f4fb8-ce71-4402-ba68-c64e768ab02d None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:07.657612 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 925/1859] 10.222.0.71 () {58 vars in 1018 bytes} [Thu Jul 21 20:40:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:07.661288 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f4fb8-ce71-4402-ba68-c64e768ab02d None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:07.667273 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754398a-6862-4217-8f59-7623a0127909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:07.668286 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754398a-6862-4217-8f59-7623a0127909 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:07.668723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754398a-6862-4217-8f59-7623a0127909 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:07.669197 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754398a-6862-4217-8f59-7623a0127909 admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:07.670181 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754398a-6862-4217-8f59-7623a0127909 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:07.670724 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754398a-6862-4217-8f59-7623a0127909 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:07.672989 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754398a-6862-4217-8f59-7623a0127909 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:07.689235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 935/1860] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:40:07.694628 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5918add-0703-4f17-afbf-e98f353b781c None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:07.700929 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754398a-6862-4217-8f59-7623a0127909 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:07.704425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-5754398a-6862-4217-8f59-7623a0127909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:07.735871 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5918add-0703-4f17-afbf-e98f353b781c None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:07.759992 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c600897-2d01-4ee5-bd0b-500af8883fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:07.762762 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c600897-2d01-4ee5-bd0b-500af8883fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:07.763064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c600897-2d01-4ee5-bd0b-500af8883fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:07.763463 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c600897-2d01-4ee5-bd0b-500af8883fe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:07.764973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754398a-6862-4217-8f59-7623a0127909 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:07.768995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c600897-2d01-4ee5-bd0b-500af8883fe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:07.771822 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c600897-2d01-4ee5-bd0b-500af8883fe5 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:07.795480 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89053a79-98a5-4199-8e35-674cc473b33b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:07.796827 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89053a79-98a5-4199-8e35-674cc473b33b service nova] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:07.797373 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89053a79-98a5-4199-8e35-674cc473b33b service nova] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:07.797867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89053a79-98a5-4199-8e35-674cc473b33b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:07.819715 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 926/1861] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:07 2022] POST /identity/v3/projects => generated 367 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:07.825064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c600897-2d01-4ee5-bd0b-500af8883fe5 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:07.835716 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89053a79-98a5-4199-8e35-674cc473b33b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:07.837697 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89053a79-98a5-4199-8e35-674cc473b33b service nova] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:07.861327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 936/1862] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:08.043911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c600897-2d01-4ee5-bd0b-500af8883fe5 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:08.067748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee281632-2848-47f2-b8a9-014d6cdf6680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:08.070438 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee281632-2848-47f2-b8a9-014d6cdf6680 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:08.073276 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee281632-2848-47f2-b8a9-014d6cdf6680 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:08.077162 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee281632-2848-47f2-b8a9-014d6cdf6680 admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:08.082619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee281632-2848-47f2-b8a9-014d6cdf6680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:08.099418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee281632-2848-47f2-b8a9-014d6cdf6680 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:08.164110 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 927/1863] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:07 2022] POST /identity/v3/users => generated 284 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:40:08.170065 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89053a79-98a5-4199-8e35-674cc473b33b service nova] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:08.189425 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89053a79-98a5-4199-8e35-674cc473b33b service nova] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:08.224363 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c795ac0-6c61-43c8-b081-7ca811f8e459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:08.225022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c795ac0-6c61-43c8-b081-7ca811f8e459 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:08.228057 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c795ac0-6c61-43c8-b081-7ca811f8e459 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:08.228551 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c795ac0-6c61-43c8-b081-7ca811f8e459 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:08.230585 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c795ac0-6c61-43c8-b081-7ca811f8e459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:08.238330 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c795ac0-6c61-43c8-b081-7ca811f8e459 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:08.267165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 937/1864] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:08.275362 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee281632-2848-47f2-b8a9-014d6cdf6680 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:08.342697 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee281632-2848-47f2-b8a9-014d6cdf6680 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:08.354623 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9d7f0-8f1f-4ca3-943f-c943717fb1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:08.355337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9d7f0-8f1f-4ca3-943f-c943717fb1fd admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:08.355627 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9d7f0-8f1f-4ca3-943f-c943717fb1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:08.356679 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9d7f0-8f1f-4ca3-943f-c943717fb1fd admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:08.356901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad9d7f0-8f1f-4ca3-943f-c943717fb1fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:08.358543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad9d7f0-8f1f-4ca3-943f-c943717fb1fd admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:08.399611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 928/1865] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:08.404931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c795ac0-6c61-43c8-b081-7ca811f8e459 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:08.420954 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c795ac0-6c61-43c8-b081-7ca811f8e459 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:08.446591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7956edb4-b635-43be-aab8-87a41b085070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:08.447533 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956edb4-b635-43be-aab8-87a41b085070 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:08.447900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956edb4-b635-43be-aab8-87a41b085070 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:08.448152 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956edb4-b635-43be-aab8-87a41b085070 admin admin] PATH_INFO: `/v3/projects/16ce89c05c9546b4b342d366f9732271/users/5ff3f26d914246b3a06710664491b163/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:08.478119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7956edb4-b635-43be-aab8-87a41b085070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16ce89c05c9546b4b342d366f9732271, user_id=5ff3f26d914246b3a06710664491b163, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:08.481583 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7956edb4-b635-43be-aab8-87a41b085070 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:08.537318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 938/1866] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:08 2022] PUT /identity/v3/projects/16ce89c05c9546b4b342d366f9732271/users/5ff3f26d914246b3a06710664491b163/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:08.538920 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce506bbe-d0bd-4fad-aa3f-c363703fd107 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:08.539213 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce506bbe-d0bd-4fad-aa3f-c363703fd107 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:08.539497 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce506bbe-d0bd-4fad-aa3f-c363703fd107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:08.606462 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-ce506bbe-d0bd-4fad-aa3f-c363703fd107 None None] MFA Rules not processed for user `5ff3f26d914246b3a06710664491b163`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:08.608717 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-ce506bbe-d0bd-4fad-aa3f-c363703fd107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:08.624824 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:08.748888 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7956edb4-b635-43be-aab8-87a41b085070 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:08.751457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-7956edb4-b635-43be-aab8-87a41b085070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:08.830128 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 929/1867] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:08 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:08.836049 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce506bbe-d0bd-4fad-aa3f-c363703fd107 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:08.866927 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce506bbe-d0bd-4fad-aa3f-c363703fd107 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:08.880404 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414766b-9407-4d3b-bbdb-a3ef575cef96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:08.881658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414766b-9407-4d3b-bbdb-a3ef575cef96 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:08.882042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414766b-9407-4d3b-bbdb-a3ef575cef96 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:08.882422 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414766b-9407-4d3b-bbdb-a3ef575cef96 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:08.885113 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414766b-9407-4d3b-bbdb-a3ef575cef96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:08.889357 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414766b-9407-4d3b-bbdb-a3ef575cef96 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:08.900649 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7956edb4-b635-43be-aab8-87a41b085070 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:08.920612 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c2f74-c61a-4f10-b896-69e34fbb9d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:08.922215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c2f74-c61a-4f10-b896-69e34fbb9d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:08.923702 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c2f74-c61a-4f10-b896-69e34fbb9d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:08.923702 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c2f74-c61a-4f10-b896-69e34fbb9d64 admin admin] PATH_INFO: `/v3/users/be4292b5f7c5414997cb1277968a8bc6` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:08.932641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c2f74-c61a-4f10-b896-69e34fbb9d64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be4292b5f7c5414997cb1277968a8bc6)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:08.940061 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c2f74-c61a-4f10-b896-69e34fbb9d64 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:08.960107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 930/1868] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:08 2022] POST /identity/v3/projects => generated 369 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:08.962953 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414766b-9407-4d3b-bbdb-a3ef575cef96 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:09.105039 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:09.114826 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-bb9c2f74-c61a-4f10-b896-69e34fbb9d64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be4292b5f7c5414997cb1277968a8bc6'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:09.133858 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-bb9c2f74-c61a-4f10-b896-69e34fbb9d64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be4292b5f7c5414997cb1277968a8bc6'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:09.136305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-bb9c2f74-c61a-4f10-b896-69e34fbb9d64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be4292b5f7c5414997cb1277968a8bc6'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:09.169316 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-bb9c2f74-c61a-4f10-b896-69e34fbb9d64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be4292b5f7c5414997cb1277968a8bc6'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:09.183482 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414766b-9407-4d3b-bbdb-a3ef575cef96 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:09.191101 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-bb9c2f74-c61a-4f10-b896-69e34fbb9d64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be4292b5f7c5414997cb1277968a8bc6'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:09.210363 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9b6fd-2266-40d0-b2b9-4c93f116b19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:09.214122 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 939/1869] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:08 2022] DELETE /identity/v3/users/be4292b5f7c5414997cb1277968a8bc6 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:09.216558 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a1f3e-a025-475e-a7b3-88c3f20561f4 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:09.216921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a1f3e-a025-475e-a7b3-88c3f20561f4 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:09.217448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a1f3e-a025-475e-a7b3-88c3f20561f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:09.220051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9b6fd-2266-40d0-b2b9-4c93f116b19e admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:09.221436 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9b6fd-2266-40d0-b2b9-4c93f116b19e admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:09.223155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9b6fd-2266-40d0-b2b9-4c93f116b19e admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:09.224406 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9b6fd-2266-40d0-b2b9-4c93f116b19e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:09.228733 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9b6fd-2266-40d0-b2b9-4c93f116b19e admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:09.281415 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 931/1870] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:08 2022] POST /identity/v3/users => generated 291 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:09.282686 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9b6fd-2266-40d0-b2b9-4c93f116b19e admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:09.284384 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a8e9b6fd-2266-40d0-b2b9-4c93f116b19e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:09.290265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-3d4a1f3e-a025-475e-a7b3-88c3f20561f4 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:09.291509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-3d4a1f3e-a025-475e-a7b3-88c3f20561f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:09.299625 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9b6fd-2266-40d0-b2b9-4c93f116b19e admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:09.380072 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a27f86-32af-4ed1-934a-261400b02f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:09.382242 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a27f86-32af-4ed1-934a-261400b02f67 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:09.382531 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a27f86-32af-4ed1-934a-261400b02f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:09.385545 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a27f86-32af-4ed1-934a-261400b02f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:09.386389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a27f86-32af-4ed1-934a-261400b02f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:09.392381 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a27f86-32af-4ed1-934a-261400b02f67 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:09.429200 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 940/1871] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:09.439707 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a1f3e-a025-475e-a7b3-88c3f20561f4 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:09.445601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 932/1872] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:40:09.446601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a27f86-32af-4ed1-934a-261400b02f67 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:09.456412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-c1a27f86-32af-4ed1-934a-261400b02f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:09.477093 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a1f3e-a025-475e-a7b3-88c3f20561f4 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:09.489628 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79d1e9-f597-48f6-a352-fe9dd6fda2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:09.490712 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79d1e9-f597-48f6-a352-fe9dd6fda2be admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:09.491172 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79d1e9-f597-48f6-a352-fe9dd6fda2be admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:09.491637 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79d1e9-f597-48f6-a352-fe9dd6fda2be admin admin] PATH_INFO: `/v3/projects/36f6f6e89aa94f9a974c2ba8a74d0542/users/1ca36e536cfb4cc5ac422f4a7d1039b6/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:09.501260 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:09.567847 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79d1e9-f597-48f6-a352-fe9dd6fda2be admin admin] RBAC: Authorizing `identity:create_grant(project_id=36f6f6e89aa94f9a974c2ba8a74d0542, user_id=1ca36e536cfb4cc5ac422f4a7d1039b6, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:09.571030 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79d1e9-f597-48f6-a352-fe9dd6fda2be admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:09.608635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a27f86-32af-4ed1-934a-261400b02f67 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:09.618236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 941/1873] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:09 2022] PUT /identity/v3/projects/36f6f6e89aa94f9a974c2ba8a74d0542/users/1ca36e536cfb4cc5ac422f4a7d1039b6/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:09.624609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79d1e9-f597-48f6-a352-fe9dd6fda2be admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:09.703407 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e299d6-4659-4add-bc6f-f6c56af656b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:09.704181 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e299d6-4659-4add-bc6f-f6c56af656b6 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:09.704555 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e299d6-4659-4add-bc6f-f6c56af656b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:09.704903 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e299d6-4659-4add-bc6f-f6c56af656b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:09.764992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79d1e9-f597-48f6-a352-fe9dd6fda2be admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:09.773028 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3f372-c0c0-4305-a055-073019383ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:09.774563 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3f372-c0c0-4305-a055-073019383ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:09.774920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3f372-c0c0-4305-a055-073019383ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:09.775293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3f372-c0c0-4305-a055-073019383ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:09.776610 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b3f372-c0c0-4305-a055-073019383ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:09.779470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b3f372-c0c0-4305-a055-073019383ea8 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:09.795111 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e299d6-4659-4add-bc6f-f6c56af656b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:09.797202 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e299d6-4659-4add-bc6f-f6c56af656b6 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:09.814315 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 942/1874] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:40:09.814508 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 933/1875] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:09.819247 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3f372-c0c0-4305-a055-073019383ea8 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:09.831894 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3f372-c0c0-4305-a055-073019383ea8 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:09.837290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b99129-c9c1-4538-9e1a-cd999a8fcfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:09.837994 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b99129-c9c1-4538-9e1a-cd999a8fcfb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:09.838484 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b99129-c9c1-4538-9e1a-cd999a8fcfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:09.838769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b99129-c9c1-4538-9e1a-cd999a8fcfb7 admin admin] PATH_INFO: `/v3/projects/36f6f6e89aa94f9a974c2ba8a74d0542/users/1ca36e536cfb4cc5ac422f4a7d1039b6/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:09.841511 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b99129-c9c1-4538-9e1a-cd999a8fcfb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36f6f6e89aa94f9a974c2ba8a74d0542, user_id=1ca36e536cfb4cc5ac422f4a7d1039b6, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:09.844267 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b99129-c9c1-4538-9e1a-cd999a8fcfb7 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:09.857126 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 943/1876] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:09 2022] PUT /identity/v3/projects/36f6f6e89aa94f9a974c2ba8a74d0542/users/1ca36e536cfb4cc5ac422f4a7d1039b6/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:09.869061 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23847a-f8d1-45ba-8469-219101b51aec None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:09.870129 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23847a-f8d1-45ba-8469-219101b51aec None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:09.871339 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23847a-f8d1-45ba-8469-219101b51aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:09.904612 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-7e23847a-f8d1-45ba-8469-219101b51aec None None] MFA Rules not processed for user `1ca36e536cfb4cc5ac422f4a7d1039b6`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:09.905934 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-7e23847a-f8d1-45ba-8469-219101b51aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:10.004988 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 934/1877] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:09 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:10.020806 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b99129-c9c1-4538-9e1a-cd999a8fcfb7 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:10.023683 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-79b99129-c9c1-4538-9e1a-cd999a8fcfb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:10.067012 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:10.123605 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b99129-c9c1-4538-9e1a-cd999a8fcfb7 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:10.127851 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b4727-d9f9-467d-849c-fbb67932a0a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:10.128792 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b4727-d9f9-467d-849c-fbb67932a0a5 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:10.129172 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b4727-d9f9-467d-849c-fbb67932a0a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:10.129502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b4727-d9f9-467d-849c-fbb67932a0a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:10.143275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b4727-d9f9-467d-849c-fbb67932a0a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:10.144207 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b4727-d9f9-467d-849c-fbb67932a0a5 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:10.155883 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 944/1878] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:10 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:10.320395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e23847a-f8d1-45ba-8469-219101b51aec None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:10.323585 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-7e23847a-f8d1-45ba-8469-219101b51aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:10.340511 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:10.467214 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e23847a-f8d1-45ba-8469-219101b51aec None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:10.473569 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dce284-a50b-4daa-a6de-ed8e5578858d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:10.475093 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dce284-a50b-4daa-a6de-ed8e5578858d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:10.475586 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dce284-a50b-4daa-a6de-ed8e5578858d admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:10.476082 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dce284-a50b-4daa-a6de-ed8e5578858d admin admin] PATH_INFO: `/v3/projects/0fd5f130eda146eba0585d6d348b204b` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:10.479748 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dce284-a50b-4daa-a6de-ed8e5578858d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fd5f130eda146eba0585d6d348b204b)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:10.483629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dce284-a50b-4daa-a6de-ed8e5578858d admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:10.571145 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-d1dce284-a50b-4daa-a6de-ed8e5578858d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fd5f130eda146eba0585d6d348b204b', 'request_id': 'req-d1dce284-a50b-4daa-a6de-ed8e5578858d'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:10.588344 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-d1dce284-a50b-4daa-a6de-ed8e5578858d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fd5f130eda146eba0585d6d348b204b', 'request_id': 'req-d1dce284-a50b-4daa-a6de-ed8e5578858d'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:10.601349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-d1dce284-a50b-4daa-a6de-ed8e5578858d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fd5f130eda146eba0585d6d348b204b', 'request_id': 'req-d1dce284-a50b-4daa-a6de-ed8e5578858d'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:10.631582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-d1dce284-a50b-4daa-a6de-ed8e5578858d admin admin] The token cache is being invalidate because project 0fd5f130eda146eba0585d6d348b204b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:10.632259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-d1dce284-a50b-4daa-a6de-ed8e5578858d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:10.633603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-d1dce284-a50b-4daa-a6de-ed8e5578858d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:10.636606 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 935/1879] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:10 2022] DELETE /identity/v3/projects/0fd5f130eda146eba0585d6d348b204b => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:10.643313 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b4727-d9f9-467d-849c-fbb67932a0a5 service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:10.662046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b4727-d9f9-467d-849c-fbb67932a0a5 service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:10.766488 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4261ebb5-d080-46c1-bf8d-0c16f1a9e7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:10.767428 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261ebb5-d080-46c1-bf8d-0c16f1a9e7ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:10.767834 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261ebb5-d080-46c1-bf8d-0c16f1a9e7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:10.768243 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261ebb5-d080-46c1-bf8d-0c16f1a9e7ab admin admin] PATH_INFO: `/v3/users/14cf7ccd9f1c4a25a123c9d329681818` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:10.769678 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4261ebb5-d080-46c1-bf8d-0c16f1a9e7ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=14cf7ccd9f1c4a25a123c9d329681818)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:10.771643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4261ebb5-d080-46c1-bf8d-0c16f1a9e7ab admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:10.841239 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4261ebb5-d080-46c1-bf8d-0c16f1a9e7ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14cf7ccd9f1c4a25a123c9d329681818'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:10.853957 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4261ebb5-d080-46c1-bf8d-0c16f1a9e7ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14cf7ccd9f1c4a25a123c9d329681818'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:10.854746 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4261ebb5-d080-46c1-bf8d-0c16f1a9e7ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14cf7ccd9f1c4a25a123c9d329681818'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:10.878071 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4261ebb5-d080-46c1-bf8d-0c16f1a9e7ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14cf7ccd9f1c4a25a123c9d329681818'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:10.889483 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4261ebb5-d080-46c1-bf8d-0c16f1a9e7ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14cf7ccd9f1c4a25a123c9d329681818'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:10.899890 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 945/1880] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:10 2022] DELETE /identity/v3/users/14cf7ccd9f1c4a25a123c9d329681818 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:11.573855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dce284-a50b-4daa-a6de-ed8e5578858d admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:11.575441 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-d1dce284-a50b-4daa-a6de-ed8e5578858d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:11.590909 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dce284-a50b-4daa-a6de-ed8e5578858d admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:11.680118 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622af4be-b16a-43d5-bd9d-e6f0b8e8605d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:11.680995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622af4be-b16a-43d5-bd9d-e6f0b8e8605d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:11.681309 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622af4be-b16a-43d5-bd9d-e6f0b8e8605d admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:11.681583 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622af4be-b16a-43d5-bd9d-e6f0b8e8605d admin admin] PATH_INFO: `/v3/projects/745051242e914579b3e15cc48a5d3a90` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:11.683362 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622af4be-b16a-43d5-bd9d-e6f0b8e8605d admin admin] RBAC: Authorizing `identity:delete_project(project_id=745051242e914579b3e15cc48a5d3a90)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:11.685380 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622af4be-b16a-43d5-bd9d-e6f0b8e8605d admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:11.740554 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:11.755575 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-622af4be-b16a-43d5-bd9d-e6f0b8e8605d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '745051242e914579b3e15cc48a5d3a90', 'request_id': 'req-622af4be-b16a-43d5-bd9d-e6f0b8e8605d'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:11.769231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-622af4be-b16a-43d5-bd9d-e6f0b8e8605d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '745051242e914579b3e15cc48a5d3a90', 'request_id': 'req-622af4be-b16a-43d5-bd9d-e6f0b8e8605d'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:11.778608 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-622af4be-b16a-43d5-bd9d-e6f0b8e8605d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '745051242e914579b3e15cc48a5d3a90', 'request_id': 'req-622af4be-b16a-43d5-bd9d-e6f0b8e8605d'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:11.802613 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-622af4be-b16a-43d5-bd9d-e6f0b8e8605d admin admin] The token cache is being invalidate because project 745051242e914579b3e15cc48a5d3a90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:11.803124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-622af4be-b16a-43d5-bd9d-e6f0b8e8605d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:11.804449 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-622af4be-b16a-43d5-bd9d-e6f0b8e8605d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:11.807387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 936/1881] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:11 2022] DELETE /identity/v3/projects/745051242e914579b3e15cc48a5d3a90 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:11.816456 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1057ed2-962f-42fc-bc5d-bc8fee6ab608 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:11.818714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1057ed2-962f-42fc-bc5d-bc8fee6ab608 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:11.820466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1057ed2-962f-42fc-bc5d-bc8fee6ab608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:11.858451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-c1057ed2-962f-42fc-bc5d-bc8fee6ab608 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:11.860677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c1057ed2-962f-42fc-bc5d-bc8fee6ab608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:11.930656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:11.979147 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 946/1882] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:11.990495 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb1b9d-40c8-4fef-86ad-cf99383ab635 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:11.990855 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb1b9d-40c8-4fef-86ad-cf99383ab635 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:11.992039 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb1b9d-40c8-4fef-86ad-cf99383ab635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:12.014438 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-0deb1b9d-40c8-4fef-86ad-cf99383ab635 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:12.016373 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-0deb1b9d-40c8-4fef-86ad-cf99383ab635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:12.049547 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 937/1883] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:12.051939 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1057ed2-962f-42fc-bc5d-bc8fee6ab608 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:12.063229 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1057ed2-962f-42fc-bc5d-bc8fee6ab608 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:12.075967 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7da63-a1cf-4146-a59e-a9a4e5f3708c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:12.076660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7da63-a1cf-4146-a59e-a9a4e5f3708c admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:12.077158 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7da63-a1cf-4146-a59e-a9a4e5f3708c admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:12.077448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7da63-a1cf-4146-a59e-a9a4e5f3708c admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:12.078193 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7da63-a1cf-4146-a59e-a9a4e5f3708c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:12.078507 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7da63-a1cf-4146-a59e-a9a4e5f3708c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:12.080264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7da63-a1cf-4146-a59e-a9a4e5f3708c admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:12.099087 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 947/1884] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:12.100302 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb1b9d-40c8-4fef-86ad-cf99383ab635 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:12.111211 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb1b9d-40c8-4fef-86ad-cf99383ab635 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:12.116498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78df314-9f75-4c3f-bc56-d73f654ff9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:12.117314 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78df314-9f75-4c3f-bc56-d73f654ff9a8 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:12.117661 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78df314-9f75-4c3f-bc56-d73f654ff9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:12.118004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78df314-9f75-4c3f-bc56-d73f654ff9a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:12.118965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78df314-9f75-4c3f-bc56-d73f654ff9a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:12.121106 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78df314-9f75-4c3f-bc56-d73f654ff9a8 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:12.151256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 938/1885] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:12 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:12.156101 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7da63-a1cf-4146-a59e-a9a4e5f3708c admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:12.169516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7da63-a1cf-4146-a59e-a9a4e5f3708c admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:12.186259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78df314-9f75-4c3f-bc56-d73f654ff9a8 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:12.188310 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f78df314-9f75-4c3f-bc56-d73f654ff9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:12.255450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca389dc9-6639-4cf1-8f64-abcdb3c86e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:12.256145 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca389dc9-6639-4cf1-8f64-abcdb3c86e48 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:12.256431 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca389dc9-6639-4cf1-8f64-abcdb3c86e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:12.256710 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca389dc9-6639-4cf1-8f64-abcdb3c86e48 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:12.257486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca389dc9-6639-4cf1-8f64-abcdb3c86e48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:12.259540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca389dc9-6639-4cf1-8f64-abcdb3c86e48 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:12.277593 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78df314-9f75-4c3f-bc56-d73f654ff9a8 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:12.283581 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60791b-140d-43b1-8592-a5d83827108e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:12.284543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60791b-140d-43b1-8592-a5d83827108e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:12.284826 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60791b-140d-43b1-8592-a5d83827108e admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:12.285109 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 948/1886] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:12 2022] POST /identity/v3/users => generated 291 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:40:12.285240 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60791b-140d-43b1-8592-a5d83827108e admin admin] PATH_INFO: `/v3/users/5ff3f26d914246b3a06710664491b163` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:12.286369 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60791b-140d-43b1-8592-a5d83827108e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ff3f26d914246b3a06710664491b163)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:12.288919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60791b-140d-43b1-8592-a5d83827108e admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:12.289325 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca389dc9-6639-4cf1-8f64-abcdb3c86e48 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:12.303768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca389dc9-6639-4cf1-8f64-abcdb3c86e48 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:12.310059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2d9e3-d345-4ae8-9d4f-fdf8c0f5e0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:12.310749 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2d9e3-d345-4ae8-9d4f-fdf8c0f5e0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:12.311921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2d9e3-d345-4ae8-9d4f-fdf8c0f5e0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:12.312207 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2d9e3-d345-4ae8-9d4f-fdf8c0f5e0f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:12.312970 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2d9e3-d345-4ae8-9d4f-fdf8c0f5e0f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:12.314634 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2d9e3-d345-4ae8-9d4f-fdf8c0f5e0f8 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:12.362292 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2c60791b-140d-43b1-8592-a5d83827108e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ff3f26d914246b3a06710664491b163'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:12.372989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2c60791b-140d-43b1-8592-a5d83827108e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ff3f26d914246b3a06710664491b163'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:12.373754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2c60791b-140d-43b1-8592-a5d83827108e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ff3f26d914246b3a06710664491b163'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:12.377770 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 949/1887] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:40:12.384953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2d9e3-d345-4ae8-9d4f-fdf8c0f5e0f8 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:12.386833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2c60791b-140d-43b1-8592-a5d83827108e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ff3f26d914246b3a06710664491b163'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:12.387592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2c60791b-140d-43b1-8592-a5d83827108e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ff3f26d914246b3a06710664491b163'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:12.394069 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-12f2d9e3-d345-4ae8-9d4f-fdf8c0f5e0f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:12.426724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 939/1888] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:12 2022] DELETE /identity/v3/users/5ff3f26d914246b3a06710664491b163 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:12.444717 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ae331-24b1-4065-9181-464beb76bf69 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:12.446439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ae331-24b1-4065-9181-464beb76bf69 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:12.446727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ae331-24b1-4065-9181-464beb76bf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:12.504294 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2d9e3-d345-4ae8-9d4f-fdf8c0f5e0f8 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:12.523740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-546ae331-24b1-4065-9181-464beb76bf69 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:12.525945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-546ae331-24b1-4065-9181-464beb76bf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:12.589494 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:12.713346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b10da1-7e1a-423b-87df-111c7554920a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:12.714730 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b10da1-7e1a-423b-87df-111c7554920a admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:12.715331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b10da1-7e1a-423b-87df-111c7554920a admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:12.716077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b10da1-7e1a-423b-87df-111c7554920a admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:12.717604 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b10da1-7e1a-423b-87df-111c7554920a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:12.721607 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b10da1-7e1a-423b-87df-111c7554920a admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:12.740030 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 950/1889] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:12.745773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b10da1-7e1a-423b-87df-111c7554920a admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:12.761789 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b10da1-7e1a-423b-87df-111c7554920a admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:12.769016 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1d82d-edb4-48f4-b83f-0f18e3076502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:12.770467 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1d82d-edb4-48f4-b83f-0f18e3076502 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:12.770984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1d82d-edb4-48f4-b83f-0f18e3076502 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:12.773174 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 940/1890] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:12.784143 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1d82d-edb4-48f4-b83f-0f18e3076502 admin admin] PATH_INFO: `/v3/projects/6be88c2f7bd844ed9073bc8072bafef0/users/42040d50ed5d4b12bbd6f79ad2e2f531/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:12.797512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ae331-24b1-4065-9181-464beb76bf69 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:12.800820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-546ae331-24b1-4065-9181-464beb76bf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:12.812306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1d82d-edb4-48f4-b83f-0f18e3076502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6be88c2f7bd844ed9073bc8072bafef0, user_id=42040d50ed5d4b12bbd6f79ad2e2f531, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:12.814106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1d82d-edb4-48f4-b83f-0f18e3076502 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:12.837533 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 951/1891] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:12 2022] PUT /identity/v3/projects/6be88c2f7bd844ed9073bc8072bafef0/users/42040d50ed5d4b12bbd6f79ad2e2f531/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:12.850845 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8e99d-8bfd-4916-bf9c-d4163e192188 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:12.851490 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8e99d-8bfd-4916-bf9c-d4163e192188 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:12.852259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8e99d-8bfd-4916-bf9c-d4163e192188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:12.946962 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ae331-24b1-4065-9181-464beb76bf69 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:12.948044 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-81a8e99d-8bfd-4916-bf9c-d4163e192188 None None] MFA Rules not processed for user `42040d50ed5d4b12bbd6f79ad2e2f531`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:12.949499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-81a8e99d-8bfd-4916-bf9c-d4163e192188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:12.973793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:13.025228 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc9e30-0763-4668-880c-a0b25e0e396b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:13.026309 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc9e30-0763-4668-880c-a0b25e0e396b service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:13.026760 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc9e30-0763-4668-880c-a0b25e0e396b service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:13.027121 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc9e30-0763-4668-880c-a0b25e0e396b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:13.093699 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 952/1892] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:13.094974 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8e99d-8bfd-4916-bf9c-d4163e192188 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:13.118811 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc9e30-0763-4668-880c-a0b25e0e396b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:13.120605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc9e30-0763-4668-880c-a0b25e0e396b service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:13.137053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 941/1893] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:13.190798 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8e99d-8bfd-4916-bf9c-d4163e192188 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:13.196484 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fe6ca-c4d4-4f32-ae64-0966d38bcd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:13.197684 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fe6ca-c4d4-4f32-ae64-0966d38bcd07 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:13.198174 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fe6ca-c4d4-4f32-ae64-0966d38bcd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:13.198672 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fe6ca-c4d4-4f32-ae64-0966d38bcd07 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:13.199394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fe6ca-c4d4-4f32-ae64-0966d38bcd07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:13.201658 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fe6ca-c4d4-4f32-ae64-0966d38bcd07 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:13.245472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 953/1894] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:13 2022] POST /identity/v3/projects => generated 381 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:13.249965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc9e30-0763-4668-880c-a0b25e0e396b service neutron] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:13.264853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc9e30-0763-4668-880c-a0b25e0e396b service neutron] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:13.292682 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:13.372242 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285df3b0-09e9-46d6-98c4-eca8d5046374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:13.372941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285df3b0-09e9-46d6-98c4-eca8d5046374 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:13.373290 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285df3b0-09e9-46d6-98c4-eca8d5046374 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:13.373683 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285df3b0-09e9-46d6-98c4-eca8d5046374 admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:13.374312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285df3b0-09e9-46d6-98c4-eca8d5046374 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:13.376413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285df3b0-09e9-46d6-98c4-eca8d5046374 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:13.412615 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 942/1895] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:13 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:13.414838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fe6ca-c4d4-4f32-ae64-0966d38bcd07 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:13.415140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285df3b0-09e9-46d6-98c4-eca8d5046374 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:13.417042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-285df3b0-09e9-46d6-98c4-eca8d5046374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:13.428768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fe6ca-c4d4-4f32-ae64-0966d38bcd07 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:13.433811 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7fadbd-7fd3-44a2-9986-dd681a92f0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:13.434509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7fadbd-7fd3-44a2-9986-dd681a92f0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:13.434798 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7fadbd-7fd3-44a2-9986-dd681a92f0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:13.435222 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7fadbd-7fd3-44a2-9986-dd681a92f0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:13.436060 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7fadbd-7fd3-44a2-9986-dd681a92f0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:13.437687 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7fadbd-7fd3-44a2-9986-dd681a92f0b1 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:13.441916 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285df3b0-09e9-46d6-98c4-eca8d5046374 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:13.451387 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 954/1896] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:13.462435 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820ad4e-abfa-457f-acb4-4f700edaa1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:13.462652 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7fadbd-7fd3-44a2-9986-dd681a92f0b1 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:13.464041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820ad4e-abfa-457f-acb4-4f700edaa1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:13.464333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820ad4e-abfa-457f-acb4-4f700edaa1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:13.465476 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820ad4e-abfa-457f-acb4-4f700edaa1b2 admin admin] PATH_INFO: `/v3/users/ed950511eb4247c0907f4654ef0c5a6e` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:13.465959 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3820ad4e-abfa-457f-acb4-4f700edaa1b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed950511eb4247c0907f4654ef0c5a6e)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:13.468591 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3820ad4e-abfa-457f-acb4-4f700edaa1b2 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:13.475817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7fadbd-7fd3-44a2-9986-dd681a92f0b1 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:13.487318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26cf64-6abf-4d37-a0ff-050d2bbabaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:13.488401 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26cf64-6abf-4d37-a0ff-050d2bbabaca admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:13.488781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26cf64-6abf-4d37-a0ff-050d2bbabaca admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:13.489474 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26cf64-6abf-4d37-a0ff-050d2bbabaca admin admin] PATH_INFO: `/v3/projects/fad137c4b8c24a03ad9c20910d3c9a66/users/a69f749fccb74a719f4e33709caa17ae/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:13.511330 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad26cf64-6abf-4d37-a0ff-050d2bbabaca admin admin] RBAC: Authorizing `identity:create_grant(project_id=fad137c4b8c24a03ad9c20910d3c9a66, user_id=a69f749fccb74a719f4e33709caa17ae, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:13.515069 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad26cf64-6abf-4d37-a0ff-050d2bbabaca admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:13.534293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 955/1897] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:13 2022] PUT /identity/v3/projects/fad137c4b8c24a03ad9c20910d3c9a66/users/a69f749fccb74a719f4e33709caa17ae/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:13.537983 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26cf64-6abf-4d37-a0ff-050d2bbabaca admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:13.543420 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3820ad4e-abfa-457f-acb4-4f700edaa1b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed950511eb4247c0907f4654ef0c5a6e'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:13.554206 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3820ad4e-abfa-457f-acb4-4f700edaa1b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed950511eb4247c0907f4654ef0c5a6e'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:13.555212 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3820ad4e-abfa-457f-acb4-4f700edaa1b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed950511eb4247c0907f4654ef0c5a6e'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:13.565045 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3820ad4e-abfa-457f-acb4-4f700edaa1b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed950511eb4247c0907f4654ef0c5a6e'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:13.565920 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3820ad4e-abfa-457f-acb4-4f700edaa1b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed950511eb4247c0907f4654ef0c5a6e'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:13.601842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 943/1898] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:13 2022] DELETE /identity/v3/users/ed950511eb4247c0907f4654ef0c5a6e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:13.607077 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b907ee-43c8-4aac-b9d0-25d3e96433ee None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:13.607568 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b907ee-43c8-4aac-b9d0-25d3e96433ee None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:13.607913 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b907ee-43c8-4aac-b9d0-25d3e96433ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:13.626165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26cf64-6abf-4d37-a0ff-050d2bbabaca admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:13.629937 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-c2b907ee-43c8-4aac-b9d0-25d3e96433ee None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:13.631351 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-c2b907ee-43c8-4aac-b9d0-25d3e96433ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:13.635334 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:13.711788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 944/1899] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:13.711996 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad9d47-2538-4414-ac27-4761bc718824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:13.712577 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad9d47-2538-4414-ac27-4761bc718824 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:13.713146 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad9d47-2538-4414-ac27-4761bc718824 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:13.714121 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad9d47-2538-4414-ac27-4761bc718824 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:13.714310 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b907ee-43c8-4aac-b9d0-25d3e96433ee None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:13.715027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad9d47-2538-4414-ac27-4761bc718824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:13.716755 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad9d47-2538-4414-ac27-4761bc718824 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:13.718399 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-c2b907ee-43c8-4aac-b9d0-25d3e96433ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:13.730874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 956/1900] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:40:13.731844 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad9d47-2538-4414-ac27-4761bc718824 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:13.733809 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-abad9d47-2538-4414-ac27-4761bc718824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:13.736668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b907ee-43c8-4aac-b9d0-25d3e96433ee None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:13.742071 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aadfd1-8583-4c2a-ab8b-12fc5409c331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:13.742954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aadfd1-8583-4c2a-ab8b-12fc5409c331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:13.743237 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aadfd1-8583-4c2a-ab8b-12fc5409c331 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:13.743515 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aadfd1-8583-4c2a-ab8b-12fc5409c331 admin admin] PATH_INFO: `/v3/projects/16ce89c05c9546b4b342d366f9732271` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:13.745223 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aadfd1-8583-4c2a-ab8b-12fc5409c331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16ce89c05c9546b4b342d366f9732271)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:13.747258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aadfd1-8583-4c2a-ab8b-12fc5409c331 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:13.769435 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:13.798466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-53aadfd1-8583-4c2a-ab8b-12fc5409c331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16ce89c05c9546b4b342d366f9732271', 'request_id': 'req-53aadfd1-8583-4c2a-ab8b-12fc5409c331'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:13.799758 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad9d47-2538-4414-ac27-4761bc718824 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:13.806003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-53aadfd1-8583-4c2a-ab8b-12fc5409c331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16ce89c05c9546b4b342d366f9732271', 'request_id': 'req-53aadfd1-8583-4c2a-ab8b-12fc5409c331'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:13.813847 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-53aadfd1-8583-4c2a-ab8b-12fc5409c331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16ce89c05c9546b4b342d366f9732271', 'request_id': 'req-53aadfd1-8583-4c2a-ab8b-12fc5409c331'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:13.826455 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-53aadfd1-8583-4c2a-ab8b-12fc5409c331 admin admin] The token cache is being invalidate because project 16ce89c05c9546b4b342d366f9732271 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:13.826749 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-53aadfd1-8583-4c2a-ab8b-12fc5409c331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:13.827575 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-53aadfd1-8583-4c2a-ab8b-12fc5409c331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:13.835118 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 945/1901] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:13 2022] DELETE /identity/v3/projects/16ce89c05c9546b4b342d366f9732271 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:13.835118 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aadfd1-8583-4c2a-ab8b-12fc5409c331 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:13.836787 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-53aadfd1-8583-4c2a-ab8b-12fc5409c331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:13.860782 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef1a18-9d6b-48a5-a9f8-a6ef743a949f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:13.861522 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef1a18-9d6b-48a5-a9f8-a6ef743a949f service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:13.861838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef1a18-9d6b-48a5-a9f8-a6ef743a949f service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:13.862125 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef1a18-9d6b-48a5-a9f8-a6ef743a949f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:13.864012 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-2fef1a18-9d6b-48a5-a9f8-a6ef743a949f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:13.911805 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aadfd1-8583-4c2a-ab8b-12fc5409c331 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:13.921198 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754efe9-802c-4101-b86b-e2e4466b941f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:13.921198 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754efe9-802c-4101-b86b-e2e4466b941f admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:13.921198 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754efe9-802c-4101-b86b-e2e4466b941f admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:13.921419 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754efe9-802c-4101-b86b-e2e4466b941f admin admin] PATH_INFO: `/v3/projects/fad137c4b8c24a03ad9c20910d3c9a66/users/a69f749fccb74a719f4e33709caa17ae/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:13.922919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754efe9-802c-4101-b86b-e2e4466b941f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fad137c4b8c24a03ad9c20910d3c9a66, user_id=a69f749fccb74a719f4e33709caa17ae, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:13.925674 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754efe9-802c-4101-b86b-e2e4466b941f admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:13.937011 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 946/1902] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:13 2022] PUT /identity/v3/projects/fad137c4b8c24a03ad9c20910d3c9a66/users/a69f749fccb74a719f4e33709caa17ae/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:13.937876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754efe9-802c-4101-b86b-e2e4466b941f admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:13.939628 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-0754efe9-802c-4101-b86b-e2e4466b941f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:13.941120 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef1a18-9d6b-48a5-a9f8-a6ef743a949f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:13.942920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef1a18-9d6b-48a5-a9f8-a6ef743a949f service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:13.956470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 957/1903] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:13.957927 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e90427-d3ca-4614-bd60-f398175946a2 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:13.958350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e90427-d3ca-4614-bd60-f398175946a2 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:13.958752 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e90427-d3ca-4614-bd60-f398175946a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:13.982291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:13.992591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-b0e90427-d3ca-4614-bd60-f398175946a2 None None] MFA Rules not processed for user `a69f749fccb74a719f4e33709caa17ae`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:13.994135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-b0e90427-d3ca-4614-bd60-f398175946a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:14.059950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754efe9-802c-4101-b86b-e2e4466b941f admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:14.065537 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c555e-5252-4625-bafa-0374d6891139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:14.066857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c555e-5252-4625-bafa-0374d6891139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:14.066974 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c555e-5252-4625-bafa-0374d6891139 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:14.067232 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c555e-5252-4625-bafa-0374d6891139 admin admin] PATH_INFO: `/v3/users/1ca36e536cfb4cc5ac422f4a7d1039b6` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:14.068828 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c555e-5252-4625-bafa-0374d6891139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ca36e536cfb4cc5ac422f4a7d1039b6)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:14.070977 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c555e-5252-4625-bafa-0374d6891139 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:14.111037 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 958/1904] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:14.121851 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e90427-d3ca-4614-bd60-f398175946a2 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:14.128890 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-b0e90427-d3ca-4614-bd60-f398175946a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:14.172136 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:14.180901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-056c555e-5252-4625-bafa-0374d6891139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ca36e536cfb4cc5ac422f4a7d1039b6'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:14.197209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-056c555e-5252-4625-bafa-0374d6891139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ca36e536cfb4cc5ac422f4a7d1039b6'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:14.198975 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-056c555e-5252-4625-bafa-0374d6891139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ca36e536cfb4cc5ac422f4a7d1039b6'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:14.214925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-056c555e-5252-4625-bafa-0374d6891139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ca36e536cfb4cc5ac422f4a7d1039b6'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:14.216271 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-056c555e-5252-4625-bafa-0374d6891139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ca36e536cfb4cc5ac422f4a7d1039b6'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:14.277063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 947/1905] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:13 2022] DELETE /identity/v3/users/1ca36e536cfb4cc5ac422f4a7d1039b6 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:14.287517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e90427-d3ca-4614-bd60-f398175946a2 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:14.367385 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe6b30-de75-467e-8201-04c9ac7ab25d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:14.368103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe6b30-de75-467e-8201-04c9ac7ab25d service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:14.368477 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe6b30-de75-467e-8201-04c9ac7ab25d service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:14.368849 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe6b30-de75-467e-8201-04c9ac7ab25d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:14.371210 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-5cbe6b30-de75-467e-8201-04c9ac7ab25d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:14.536164 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe6b30-de75-467e-8201-04c9ac7ab25d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:14.539890 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe6b30-de75-467e-8201-04c9ac7ab25d service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:14.556030 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:14.570341 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 959/1906] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:14.812403 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c555e-5252-4625-bafa-0374d6891139 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:14.814479 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-056c555e-5252-4625-bafa-0374d6891139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:14.835437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe6b30-de75-467e-8201-04c9ac7ab25d service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:14.840073 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-5cbe6b30-de75-467e-8201-04c9ac7ab25d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:14.950781 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c555e-5252-4625-bafa-0374d6891139 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:14.956718 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057d76d-4fd3-4d70-a770-085429e89f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:14.957626 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057d76d-4fd3-4d70-a770-085429e89f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:14.957908 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057d76d-4fd3-4d70-a770-085429e89f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:14.958190 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057d76d-4fd3-4d70-a770-085429e89f31 admin admin] PATH_INFO: `/v3/projects/36f6f6e89aa94f9a974c2ba8a74d0542` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:14.962320 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057d76d-4fd3-4d70-a770-085429e89f31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36f6f6e89aa94f9a974c2ba8a74d0542)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:14.967635 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057d76d-4fd3-4d70-a770-085429e89f31 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:15.021271 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe6b30-de75-467e-8201-04c9ac7ab25d service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:15.032118 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ee64f-6d77-46b6-bb41-2d6efb5e4a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:15.034048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ee64f-6d77-46b6-bb41-2d6efb5e4a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:15.034901 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ee64f-6d77-46b6-bb41-2d6efb5e4a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:15.038673 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ee64f-6d77-46b6-bb41-2d6efb5e4a78 admin admin] PATH_INFO: `/v3/projects/01ca03bf6ba14cceb8d90f0255d9db85` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:15.065944 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ee64f-6d77-46b6-bb41-2d6efb5e4a78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01ca03bf6ba14cceb8d90f0255d9db85)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:15.073627 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ee64f-6d77-46b6-bb41-2d6efb5e4a78 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:15.087846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1057d76d-4fd3-4d70-a770-085429e89f31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36f6f6e89aa94f9a974c2ba8a74d0542', 'request_id': 'req-1057d76d-4fd3-4d70-a770-085429e89f31'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.101385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1057d76d-4fd3-4d70-a770-085429e89f31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36f6f6e89aa94f9a974c2ba8a74d0542', 'request_id': 'req-1057d76d-4fd3-4d70-a770-085429e89f31'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.118780 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1057d76d-4fd3-4d70-a770-085429e89f31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36f6f6e89aa94f9a974c2ba8a74d0542', 'request_id': 'req-1057d76d-4fd3-4d70-a770-085429e89f31'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.136105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1057d76d-4fd3-4d70-a770-085429e89f31 admin admin] The token cache is being invalidate because project 36f6f6e89aa94f9a974c2ba8a74d0542 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:15.136549 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1057d76d-4fd3-4d70-a770-085429e89f31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.137302 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1057d76d-4fd3-4d70-a770-085429e89f31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.142744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 948/1907] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:14 2022] DELETE /identity/v3/projects/36f6f6e89aa94f9a974c2ba8a74d0542 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:15.144433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79043889-c865-4f9c-af14-51b75f2b65ed None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:15.144736 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79043889-c865-4f9c-af14-51b75f2b65ed None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:15.144966 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79043889-c865-4f9c-af14-51b75f2b65ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:15.164846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f49ee64f-6d77-46b6-bb41-2d6efb5e4a78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01ca03bf6ba14cceb8d90f0255d9db85', 'request_id': 'req-f49ee64f-6d77-46b6-bb41-2d6efb5e4a78'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.168311 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-79043889-c865-4f9c-af14-51b75f2b65ed None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:15.170116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-79043889-c865-4f9c-af14-51b75f2b65ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:15.175501 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f49ee64f-6d77-46b6-bb41-2d6efb5e4a78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01ca03bf6ba14cceb8d90f0255d9db85', 'request_id': 'req-f49ee64f-6d77-46b6-bb41-2d6efb5e4a78'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.186308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f49ee64f-6d77-46b6-bb41-2d6efb5e4a78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01ca03bf6ba14cceb8d90f0255d9db85', 'request_id': 'req-f49ee64f-6d77-46b6-bb41-2d6efb5e4a78'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.199165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f49ee64f-6d77-46b6-bb41-2d6efb5e4a78 admin admin] The token cache is being invalidate because project 01ca03bf6ba14cceb8d90f0255d9db85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:15.199634 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f49ee64f-6d77-46b6-bb41-2d6efb5e4a78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.200326 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f49ee64f-6d77-46b6-bb41-2d6efb5e4a78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.201876 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 960/1908] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:14 2022] DELETE /identity/v3/projects/01ca03bf6ba14cceb8d90f0255d9db85 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:15.206245 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ee64f-6d77-46b6-bb41-2d6efb5e4a78 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:15.208450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-f49ee64f-6d77-46b6-bb41-2d6efb5e4a78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:15.229178 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:15.248346 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 949/1909] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:15.249296 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79043889-c865-4f9c-af14-51b75f2b65ed None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:15.250732 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-79043889-c865-4f9c-af14-51b75f2b65ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:15.271417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79043889-c865-4f9c-af14-51b75f2b65ed None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:15.276884 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5288c-1da6-47cb-a67a-b7822300e09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:15.277795 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5288c-1da6-47cb-a67a-b7822300e09c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:15.278191 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5288c-1da6-47cb-a67a-b7822300e09c admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:15.278474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5288c-1da6-47cb-a67a-b7822300e09c admin admin] PATH_INFO: `/v3/users/42040d50ed5d4b12bbd6f79ad2e2f531` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:15.279705 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5288c-1da6-47cb-a67a-b7822300e09c admin admin] RBAC: Authorizing `identity:delete_user(user_id=42040d50ed5d4b12bbd6f79ad2e2f531)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:15.281788 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5288c-1da6-47cb-a67a-b7822300e09c admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:15.295683 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ee64f-6d77-46b6-bb41-2d6efb5e4a78 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:15.300561 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abcb75-43f9-4596-bb7f-f8d382d83cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:15.302518 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abcb75-43f9-4596-bb7f-f8d382d83cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:15.302518 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abcb75-43f9-4596-bb7f-f8d382d83cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:15.302518 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abcb75-43f9-4596-bb7f-f8d382d83cb3 admin admin] PATH_INFO: `/v3/users/3e5b45f9f2334e209f2acbe8d056a7f2` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:15.303840 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abcb75-43f9-4596-bb7f-f8d382d83cb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e5b45f9f2334e209f2acbe8d056a7f2)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:15.305819 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abcb75-43f9-4596-bb7f-f8d382d83cb3 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:15.339931 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1af5288c-1da6-47cb-a67a-b7822300e09c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42040d50ed5d4b12bbd6f79ad2e2f531'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.351102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1af5288c-1da6-47cb-a67a-b7822300e09c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42040d50ed5d4b12bbd6f79ad2e2f531'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.352232 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1af5288c-1da6-47cb-a67a-b7822300e09c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42040d50ed5d4b12bbd6f79ad2e2f531'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.357324 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:15.362566 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1af5288c-1da6-47cb-a67a-b7822300e09c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42040d50ed5d4b12bbd6f79ad2e2f531'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.363426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1af5288c-1da6-47cb-a67a-b7822300e09c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42040d50ed5d4b12bbd6f79ad2e2f531'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.372664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-61abcb75-43f9-4596-bb7f-f8d382d83cb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e5b45f9f2334e209f2acbe8d056a7f2'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.394792 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 950/1910] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:15 2022] DELETE /identity/v3/users/42040d50ed5d4b12bbd6f79ad2e2f531 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:15.399659 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-61abcb75-43f9-4596-bb7f-f8d382d83cb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e5b45f9f2334e209f2acbe8d056a7f2'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.399659 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-61abcb75-43f9-4596-bb7f-f8d382d83cb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e5b45f9f2334e209f2acbe8d056a7f2'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.399983 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955b91f-c690-4bae-a8fb-595964963c96 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:15.399983 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955b91f-c690-4bae-a8fb-595964963c96 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:15.399983 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955b91f-c690-4bae-a8fb-595964963c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:15.422398 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-61abcb75-43f9-4596-bb7f-f8d382d83cb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e5b45f9f2334e209f2acbe8d056a7f2'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.426988 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-d955b91f-c690-4bae-a8fb-595964963c96 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:15.428289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-d955b91f-c690-4bae-a8fb-595964963c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:15.433198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-61abcb75-43f9-4596-bb7f-f8d382d83cb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e5b45f9f2334e209f2acbe8d056a7f2'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:15.441520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 961/1911] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:15 2022] DELETE /identity/v3/users/3e5b45f9f2334e209f2acbe8d056a7f2 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:15.443916 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d684d2e-cf43-4b3f-bf20-0b343dccec39 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:15.444443 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d684d2e-cf43-4b3f-bf20-0b343dccec39 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:15.444936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d684d2e-cf43-4b3f-bf20-0b343dccec39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:15.478180 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-5d684d2e-cf43-4b3f-bf20-0b343dccec39 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:15.482312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-5d684d2e-cf43-4b3f-bf20-0b343dccec39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:15.547222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 951/1912] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:15.551767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955b91f-c690-4bae-a8fb-595964963c96 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:15.563191 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955b91f-c690-4bae-a8fb-595964963c96 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:15.601784 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 962/1913] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:15.619000 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d684d2e-cf43-4b3f-bf20-0b343dccec39 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:15.621694 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-5d684d2e-cf43-4b3f-bf20-0b343dccec39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:15.698515 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6acae53-7ed9-4f9a-893b-7b41e3784e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:15.699291 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6acae53-7ed9-4f9a-893b-7b41e3784e19 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:15.699576 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6acae53-7ed9-4f9a-893b-7b41e3784e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:15.699849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6acae53-7ed9-4f9a-893b-7b41e3784e19 admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:15.700765 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6acae53-7ed9-4f9a-893b-7b41e3784e19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:15.701103 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6acae53-7ed9-4f9a-893b-7b41e3784e19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:15.703257 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6acae53-7ed9-4f9a-893b-7b41e3784e19 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:15.717062 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d684d2e-cf43-4b3f-bf20-0b343dccec39 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:15.718712 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:15.722369 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef45e3-a0c9-492b-a9a3-aa9d7e81a249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:15.723455 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef45e3-a0c9-492b-a9a3-aa9d7e81a249 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:15.723981 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef45e3-a0c9-492b-a9a3-aa9d7e81a249 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:15.724436 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef45e3-a0c9-492b-a9a3-aa9d7e81a249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:15.731940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 952/1914] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:40:15.736022 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6acae53-7ed9-4f9a-893b-7b41e3784e19 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:15.738140 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef45e3-a0c9-492b-a9a3-aa9d7e81a249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:15.739606 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef45e3-a0c9-492b-a9a3-aa9d7e81a249 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:15.747325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6acae53-7ed9-4f9a-893b-7b41e3784e19 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:15.750337 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 963/1915] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:15.752727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1d2a0-c20c-4487-9742-c35997169b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:15.753569 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1d2a0-c20c-4487-9742-c35997169b8c admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:15.753973 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1d2a0-c20c-4487-9742-c35997169b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:15.754352 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1d2a0-c20c-4487-9742-c35997169b8c admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:15.755164 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1d2a0-c20c-4487-9742-c35997169b8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:15.757292 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1d2a0-c20c-4487-9742-c35997169b8c admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:15.787596 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 953/1916] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:15 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:15.791990 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef45e3-a0c9-492b-a9a3-aa9d7e81a249 service neutron] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:15.802823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef45e3-a0c9-492b-a9a3-aa9d7e81a249 service neutron] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:15.861546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:15.874919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d3694-5a8f-4391-b029-81a9191c7266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:15.875878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d3694-5a8f-4391-b029-81a9191c7266 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:15.876815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d3694-5a8f-4391-b029-81a9191c7266 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:15.877209 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d3694-5a8f-4391-b029-81a9191c7266 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:15.878875 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d3694-5a8f-4391-b029-81a9191c7266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:15.881408 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d3694-5a8f-4391-b029-81a9191c7266 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:15.912229 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 964/1917] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:15 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:15.916499 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1d2a0-c20c-4487-9742-c35997169b8c admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:15.933607 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1d2a0-c20c-4487-9742-c35997169b8c admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:15.934807 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d3694-5a8f-4391-b029-81a9191c7266 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:15.936848 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-af0d3694-5a8f-4391-b029-81a9191c7266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:15.947008 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ec570-5786-4f14-95d9-caa63c7d98cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:15.948440 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ec570-5786-4f14-95d9-caa63c7d98cc admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:15.948819 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ec570-5786-4f14-95d9-caa63c7d98cc admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:15.949195 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ec570-5786-4f14-95d9-caa63c7d98cc admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:15.957608 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ec570-5786-4f14-95d9-caa63c7d98cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:15.962868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ec570-5786-4f14-95d9-caa63c7d98cc admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:15.966751 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d3694-5a8f-4391-b029-81a9191c7266 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:15.975212 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ab6e8-22fc-4bce-a6cc-eaf99dd52c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:15.976367 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ab6e8-22fc-4bce-a6cc-eaf99dd52c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:15.976670 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ab6e8-22fc-4bce-a6cc-eaf99dd52c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:15.976947 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ab6e8-22fc-4bce-a6cc-eaf99dd52c02 admin admin] PATH_INFO: `/v3/projects/86547ced096442f6bec878a65e3efc98` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:15.979154 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ab6e8-22fc-4bce-a6cc-eaf99dd52c02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86547ced096442f6bec878a65e3efc98)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:15.980376 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 954/1918] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:15.984345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ec570-5786-4f14-95d9-caa63c7d98cc admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:15.985455 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ab6e8-22fc-4bce-a6cc-eaf99dd52c02 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:15.997786 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ec570-5786-4f14-95d9-caa63c7d98cc admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:16.006122 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb71f4-daa0-4621-9428-1f776dea6693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:16.006822 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb71f4-daa0-4621-9428-1f776dea6693 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:16.007107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb71f4-daa0-4621-9428-1f776dea6693 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:16.007658 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb71f4-daa0-4621-9428-1f776dea6693 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:16.008122 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb71f4-daa0-4621-9428-1f776dea6693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:16.010065 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb71f4-daa0-4621-9428-1f776dea6693 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:16.028694 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 955/1919] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:40:16.030155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb71f4-daa0-4621-9428-1f776dea6693 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:16.042716 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb71f4-daa0-4621-9428-1f776dea6693 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:16.055835 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-757ab6e8-22fc-4bce-a6cc-eaf99dd52c02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86547ced096442f6bec878a65e3efc98', 'request_id': 'req-757ab6e8-22fc-4bce-a6cc-eaf99dd52c02'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:16.068718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-757ab6e8-22fc-4bce-a6cc-eaf99dd52c02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86547ced096442f6bec878a65e3efc98', 'request_id': 'req-757ab6e8-22fc-4bce-a6cc-eaf99dd52c02'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:16.097512 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-757ab6e8-22fc-4bce-a6cc-eaf99dd52c02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86547ced096442f6bec878a65e3efc98', 'request_id': 'req-757ab6e8-22fc-4bce-a6cc-eaf99dd52c02'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:16.123025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-757ab6e8-22fc-4bce-a6cc-eaf99dd52c02 admin admin] The token cache is being invalidate because project 86547ced096442f6bec878a65e3efc98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:16.123377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-757ab6e8-22fc-4bce-a6cc-eaf99dd52c02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:16.124076 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-757ab6e8-22fc-4bce-a6cc-eaf99dd52c02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:16.129292 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 965/1920] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:15 2022] DELETE /identity/v3/projects/86547ced096442f6bec878a65e3efc98 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:16.129500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc87257-6377-45e1-8c3d-d9b72a1fa221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:16.129500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc87257-6377-45e1-8c3d-d9b72a1fa221 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:16.129500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc87257-6377-45e1-8c3d-d9b72a1fa221 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:16.129785 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc87257-6377-45e1-8c3d-d9b72a1fa221 admin admin] PATH_INFO: `/v3/projects/c548b6a7404b426d9b72a467cd2986cc/users/2b35288edc8843f1907fac2197e9c0f0/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:16.132531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36983dac-aed0-4114-8472-a62265894d57 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:16.134545 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36983dac-aed0-4114-8472-a62265894d57 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:16.135018 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36983dac-aed0-4114-8472-a62265894d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:16.150599 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc87257-6377-45e1-8c3d-d9b72a1fa221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c548b6a7404b426d9b72a467cd2986cc, user_id=2b35288edc8843f1907fac2197e9c0f0, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:16.156147 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc87257-6377-45e1-8c3d-d9b72a1fa221 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:16.172296 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-36983dac-aed0-4114-8472-a62265894d57 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:16.174373 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-36983dac-aed0-4114-8472-a62265894d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:16.174602 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 956/1921] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:16 2022] PUT /identity/v3/projects/c548b6a7404b426d9b72a467cd2986cc/users/2b35288edc8843f1907fac2197e9c0f0/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:16.186288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fea5f-2222-47a7-9ab8-2f834af02725 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:16.186683 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fea5f-2222-47a7-9ab8-2f834af02725 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:16.187038 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fea5f-2222-47a7-9ab8-2f834af02725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:16.249223 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-466fea5f-2222-47a7-9ab8-2f834af02725 None None] MFA Rules not processed for user `2b35288edc8843f1907fac2197e9c0f0`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:16.250473 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-466fea5f-2222-47a7-9ab8-2f834af02725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:16.301517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 966/1922] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:16.304199 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36983dac-aed0-4114-8472-a62265894d57 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:16.308944 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-36983dac-aed0-4114-8472-a62265894d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:16.324786 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36983dac-aed0-4114-8472-a62265894d57 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:16.332907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c52d4-3a52-4b05-a63e-baeb49e5aa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:16.333755 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c52d4-3a52-4b05-a63e-baeb49e5aa67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:16.334033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c52d4-3a52-4b05-a63e-baeb49e5aa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:16.334307 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c52d4-3a52-4b05-a63e-baeb49e5aa67 admin admin] PATH_INFO: `/v3/projects/6be88c2f7bd844ed9073bc8072bafef0` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:16.335627 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c52d4-3a52-4b05-a63e-baeb49e5aa67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6be88c2f7bd844ed9073bc8072bafef0)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:16.337769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c52d4-3a52-4b05-a63e-baeb49e5aa67 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:16.385920 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 957/1923] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:16.387287 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d12027-179c-4dc3-8420-b67f4bddb32b None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:16.387609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d12027-179c-4dc3-8420-b67f4bddb32b None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:16.387863 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d12027-179c-4dc3-8420-b67f4bddb32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:16.402998 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-176c52d4-3a52-4b05-a63e-baeb49e5aa67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6be88c2f7bd844ed9073bc8072bafef0', 'request_id': 'req-176c52d4-3a52-4b05-a63e-baeb49e5aa67'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:16.408850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-c3d12027-179c-4dc3-8420-b67f4bddb32b None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:16.410730 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-c3d12027-179c-4dc3-8420-b67f4bddb32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:16.414738 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-176c52d4-3a52-4b05-a63e-baeb49e5aa67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6be88c2f7bd844ed9073bc8072bafef0', 'request_id': 'req-176c52d4-3a52-4b05-a63e-baeb49e5aa67'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:16.421925 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=82,keystone:INSERT=5,keystone:DELETE=13 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:16.425413 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-176c52d4-3a52-4b05-a63e-baeb49e5aa67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6be88c2f7bd844ed9073bc8072bafef0', 'request_id': 'req-176c52d4-3a52-4b05-a63e-baeb49e5aa67'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:16.437728 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:16.442435 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-176c52d4-3a52-4b05-a63e-baeb49e5aa67 admin admin] The token cache is being invalidate because project 6be88c2f7bd844ed9073bc8072bafef0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:16.443315 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-176c52d4-3a52-4b05-a63e-baeb49e5aa67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:16.443927 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-176c52d4-3a52-4b05-a63e-baeb49e5aa67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:16.446002 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 967/1924] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:16 2022] DELETE /identity/v3/projects/6be88c2f7bd844ed9073bc8072bafef0 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:16.447088 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c52d4-3a52-4b05-a63e-baeb49e5aa67 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:16.448662 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-176c52d4-3a52-4b05-a63e-baeb49e5aa67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:16.488769 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 958/1925] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:16.489916 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d12027-179c-4dc3-8420-b67f4bddb32b None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:16.491358 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-c3d12027-179c-4dc3-8420-b67f4bddb32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:16.507090 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d12027-179c-4dc3-8420-b67f4bddb32b None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:16.513358 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527b301-b176-4435-88ef-a54bc486d5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:16.514268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527b301-b176-4435-88ef-a54bc486d5ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:16.514542 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527b301-b176-4435-88ef-a54bc486d5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:16.514811 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527b301-b176-4435-88ef-a54bc486d5ec admin admin] PATH_INFO: `/v3/users/a69f749fccb74a719f4e33709caa17ae` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:16.516034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527b301-b176-4435-88ef-a54bc486d5ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=a69f749fccb74a719f4e33709caa17ae)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:16.518061 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527b301-b176-4435-88ef-a54bc486d5ec admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:16.532886 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c52d4-3a52-4b05-a63e-baeb49e5aa67 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:16.537838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22629ab-38f1-45ca-b3d0-9e0089bef19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:16.538518 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22629ab-38f1-45ca-b3d0-9e0089bef19d admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:16.538804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22629ab-38f1-45ca-b3d0-9e0089bef19d admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:16.539085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22629ab-38f1-45ca-b3d0-9e0089bef19d admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:16.539798 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22629ab-38f1-45ca-b3d0-9e0089bef19d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:16.541844 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22629ab-38f1-45ca-b3d0-9e0089bef19d admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:16.568741 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3527b301-b176-4435-88ef-a54bc486d5ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a69f749fccb74a719f4e33709caa17ae'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:16.571542 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 968/1926] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:16 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:16.572445 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22629ab-38f1-45ca-b3d0-9e0089bef19d admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:16.581878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3527b301-b176-4435-88ef-a54bc486d5ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a69f749fccb74a719f4e33709caa17ae'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:16.582597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3527b301-b176-4435-88ef-a54bc486d5ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a69f749fccb74a719f4e33709caa17ae'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:16.592514 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3527b301-b176-4435-88ef-a54bc486d5ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a69f749fccb74a719f4e33709caa17ae'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:16.593182 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3527b301-b176-4435-88ef-a54bc486d5ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a69f749fccb74a719f4e33709caa17ae'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:16.618624 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 959/1927] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:16 2022] DELETE /identity/v3/users/a69f749fccb74a719f4e33709caa17ae => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:16.619591 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527b301-b176-4435-88ef-a54bc486d5ec admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:16.626201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3527b301-b176-4435-88ef-a54bc486d5ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:16.665880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22629ab-38f1-45ca-b3d0-9e0089bef19d admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:16.721930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527b301-b176-4435-88ef-a54bc486d5ec admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:16.728036 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcf41e-fc68-4cef-912e-05dda0adfd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:16.729626 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcf41e-fc68-4cef-912e-05dda0adfd74 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:16.729626 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcf41e-fc68-4cef-912e-05dda0adfd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:16.729626 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcf41e-fc68-4cef-912e-05dda0adfd74 admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:16.730381 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddcf41e-fc68-4cef-912e-05dda0adfd74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:16.732693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddcf41e-fc68-4cef-912e-05dda0adfd74 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:16.772524 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 960/1928] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:16 2022] POST /identity/v3/users => generated 300 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:16.776187 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcf41e-fc68-4cef-912e-05dda0adfd74 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:16.777569 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6f57e-154b-48cc-8d81-2a5554909181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:16.778267 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6f57e-154b-48cc-8d81-2a5554909181 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:16.778558 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6f57e-154b-48cc-8d81-2a5554909181 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:16.779098 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6f57e-154b-48cc-8d81-2a5554909181 admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:16.779730 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6f57e-154b-48cc-8d81-2a5554909181 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:16.780044 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6f57e-154b-48cc-8d81-2a5554909181 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:16.781962 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6f57e-154b-48cc-8d81-2a5554909181 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:16.796797 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcf41e-fc68-4cef-912e-05dda0adfd74 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:16.800373 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 969/1929] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:16.803308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2eca6c-37a0-4981-b740-f07b84f4319c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:16.804422 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2eca6c-37a0-4981-b740-f07b84f4319c admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:16.804911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2eca6c-37a0-4981-b740-f07b84f4319c admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:16.805599 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2eca6c-37a0-4981-b740-f07b84f4319c admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:16.806033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6f57e-154b-48cc-8d81-2a5554909181 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:16.807636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2eca6c-37a0-4981-b740-f07b84f4319c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:16.809175 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-70e6f57e-154b-48cc-8d81-2a5554909181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:16.809740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2eca6c-37a0-4981-b740-f07b84f4319c admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:16.823751 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 961/1930] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:16.827714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2eca6c-37a0-4981-b740-f07b84f4319c admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:16.832431 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6f57e-154b-48cc-8d81-2a5554909181 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:16.840981 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df22029-d6a5-42d2-92ed-cdc840c0eb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:16.842071 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df22029-d6a5-42d2-92ed-cdc840c0eb01 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:16.842506 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df22029-d6a5-42d2-92ed-cdc840c0eb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:16.842954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2eca6c-37a0-4981-b740-f07b84f4319c admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:16.843090 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df22029-d6a5-42d2-92ed-cdc840c0eb01 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:16.843878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df22029-d6a5-42d2-92ed-cdc840c0eb01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:16.846348 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df22029-d6a5-42d2-92ed-cdc840c0eb01 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:16.849832 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d6f7d-faed-456e-bbd6-fa4122ebacad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:16.850975 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d6f7d-faed-456e-bbd6-fa4122ebacad admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:16.851418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d6f7d-faed-456e-bbd6-fa4122ebacad admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:16.851882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d6f7d-faed-456e-bbd6-fa4122ebacad admin admin] PATH_INFO: `/v3/projects/57109a5ab6b741a6b734df4a2c2e1a28/users/54bc4f3872914d7ebd1dc9a3a827c7b0/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:16.865995 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:16.879772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d6f7d-faed-456e-bbd6-fa4122ebacad admin admin] RBAC: Authorizing `identity:create_grant(project_id=57109a5ab6b741a6b734df4a2c2e1a28, user_id=54bc4f3872914d7ebd1dc9a3a827c7b0, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:16.886608 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d6f7d-faed-456e-bbd6-fa4122ebacad admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:16.890232 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:16.915059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 970/1931] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:16 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:40:16.922950 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df22029-d6a5-42d2-92ed-cdc840c0eb01 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:16.930757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 962/1932] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:16 2022] PUT /identity/v3/projects/57109a5ab6b741a6b734df4a2c2e1a28/users/54bc4f3872914d7ebd1dc9a3a827c7b0/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:16.934522 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d6f7d-faed-456e-bbd6-fa4122ebacad admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:17.055082 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d6f7d-faed-456e-bbd6-fa4122ebacad admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:17.060308 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbd7ee-7ba2-42cd-880e-380b4dc63479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:17.061013 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbd7ee-7ba2-42cd-880e-380b4dc63479 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:17.061334 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbd7ee-7ba2-42cd-880e-380b4dc63479 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:17.061617 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbd7ee-7ba2-42cd-880e-380b4dc63479 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:17.062594 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fbd7ee-7ba2-42cd-880e-380b4dc63479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:17.064389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fbd7ee-7ba2-42cd-880e-380b4dc63479 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:17.066331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df22029-d6a5-42d2-92ed-cdc840c0eb01 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:17.078064 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0cb48-37bc-4867-bf14-2079a1a6591a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:17.078773 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 963/1933] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:17.079646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0cb48-37bc-4867-bf14-2079a1a6591a admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:17.080536 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0cb48-37bc-4867-bf14-2079a1a6591a admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:17.081226 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0cb48-37bc-4867-bf14-2079a1a6591a admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:17.082267 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0cb48-37bc-4867-bf14-2079a1a6591a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:17.083617 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbd7ee-7ba2-42cd-880e-380b4dc63479 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:17.084800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0cb48-37bc-4867-bf14-2079a1a6591a admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:17.095143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbd7ee-7ba2-42cd-880e-380b4dc63479 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:17.108065 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a03c0-69d4-499f-9461-08b19f3fd67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:17.110219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a03c0-69d4-499f-9461-08b19f3fd67d admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:17.110468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a03c0-69d4-499f-9461-08b19f3fd67d admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:17.111093 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a03c0-69d4-499f-9461-08b19f3fd67d admin admin] PATH_INFO: `/v3/projects/57109a5ab6b741a6b734df4a2c2e1a28/users/54bc4f3872914d7ebd1dc9a3a827c7b0/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:17.115311 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a03c0-69d4-499f-9461-08b19f3fd67d admin admin] RBAC: Authorizing `identity:create_grant(project_id=57109a5ab6b741a6b734df4a2c2e1a28, user_id=54bc4f3872914d7ebd1dc9a3a827c7b0, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:17.120274 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a03c0-69d4-499f-9461-08b19f3fd67d admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:17.129728 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 971/1934] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:16 2022] POST /identity/v3/users => generated 290 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:17.136899 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0cb48-37bc-4867-bf14-2079a1a6591a admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:17.146110 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 964/1935] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:17 2022] PUT /identity/v3/projects/57109a5ab6b741a6b734df4a2c2e1a28/users/54bc4f3872914d7ebd1dc9a3a827c7b0/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:17.158440 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0cb48-37bc-4867-bf14-2079a1a6591a admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:17.159756 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cf09e-c964-4e1e-a2b9-427be49f43e9 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:17.159983 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cf09e-c964-4e1e-a2b9-427be49f43e9 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:17.160859 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cf09e-c964-4e1e-a2b9-427be49f43e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:17.201122 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-b42cf09e-c964-4e1e-a2b9-427be49f43e9 None None] MFA Rules not processed for user `54bc4f3872914d7ebd1dc9a3a827c7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:17.202784 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-b42cf09e-c964-4e1e-a2b9-427be49f43e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:17.299114 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ee1e2-5720-49a8-bfe2-79878a1961a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:17.299932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ee1e2-5720-49a8-bfe2-79878a1961a3 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:17.300440 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ee1e2-5720-49a8-bfe2-79878a1961a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:17.300735 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ee1e2-5720-49a8-bfe2-79878a1961a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:17.301680 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ee1e2-5720-49a8-bfe2-79878a1961a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:17.304599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ee1e2-5720-49a8-bfe2-79878a1961a3 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:17.324902 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 972/1936] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:17.326344 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ee1e2-5720-49a8-bfe2-79878a1961a3 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:17.328379 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-dc4ee1e2-5720-49a8-bfe2-79878a1961a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:17.352338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ee1e2-5720-49a8-bfe2-79878a1961a3 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:17.363934 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfbbf88-4e2e-403a-bdac-d5497be82e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:17.365638 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbbf88-4e2e-403a-bdac-d5497be82e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:17.365933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbbf88-4e2e-403a-bdac-d5497be82e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:17.366210 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbbf88-4e2e-403a-bdac-d5497be82e48 admin admin] PATH_INFO: `/v3/projects/fad137c4b8c24a03ad9c20910d3c9a66` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:17.367831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfbbf88-4e2e-403a-bdac-d5497be82e48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fad137c4b8c24a03ad9c20910d3c9a66)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:17.370799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfbbf88-4e2e-403a-bdac-d5497be82e48 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:17.372013 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 965/1937] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:17.374329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cf09e-c964-4e1e-a2b9-427be49f43e9 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:17.391220 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cf09e-c964-4e1e-a2b9-427be49f43e9 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:17.397741 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63019379-798e-44a8-9a0f-751aeb4d03fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:17.398565 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63019379-798e-44a8-9a0f-751aeb4d03fb admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:17.398853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63019379-798e-44a8-9a0f-751aeb4d03fb admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:17.399199 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63019379-798e-44a8-9a0f-751aeb4d03fb admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:17.399954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63019379-798e-44a8-9a0f-751aeb4d03fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:17.402445 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63019379-798e-44a8-9a0f-751aeb4d03fb admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:17.421343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 966/1938] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:17.422393 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63019379-798e-44a8-9a0f-751aeb4d03fb admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:17.424346 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-63019379-798e-44a8-9a0f-751aeb4d03fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:17.452807 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-9dfbbf88-4e2e-403a-bdac-d5497be82e48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fad137c4b8c24a03ad9c20910d3c9a66', 'request_id': 'req-9dfbbf88-4e2e-403a-bdac-d5497be82e48'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:17.465676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-9dfbbf88-4e2e-403a-bdac-d5497be82e48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fad137c4b8c24a03ad9c20910d3c9a66', 'request_id': 'req-9dfbbf88-4e2e-403a-bdac-d5497be82e48'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:17.480944 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-9dfbbf88-4e2e-403a-bdac-d5497be82e48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fad137c4b8c24a03ad9c20910d3c9a66', 'request_id': 'req-9dfbbf88-4e2e-403a-bdac-d5497be82e48'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:17.496427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-9dfbbf88-4e2e-403a-bdac-d5497be82e48 admin admin] The token cache is being invalidate because project fad137c4b8c24a03ad9c20910d3c9a66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:17.496746 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-9dfbbf88-4e2e-403a-bdac-d5497be82e48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:17.497544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-9dfbbf88-4e2e-403a-bdac-d5497be82e48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:17.499418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 973/1939] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:17 2022] DELETE /identity/v3/projects/fad137c4b8c24a03ad9c20910d3c9a66 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:17.500232 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfbbf88-4e2e-403a-bdac-d5497be82e48 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:17.502090 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-9dfbbf88-4e2e-403a-bdac-d5497be82e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:17.504142 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63019379-798e-44a8-9a0f-751aeb4d03fb admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:17.507683 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:17.509405 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:17.575536 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e33bd-25b4-4f4d-816f-ef6326a78bf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:17.576597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e33bd-25b4-4f4d-816f-ef6326a78bf1 service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:17.576892 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e33bd-25b4-4f4d-816f-ef6326a78bf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:17.577387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e33bd-25b4-4f4d-816f-ef6326a78bf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:17.579454 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-773e33bd-25b4-4f4d-816f-ef6326a78bf1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:17.606523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfbbf88-4e2e-403a-bdac-d5497be82e48 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:17.611179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d87f2e-a348-4fff-89af-40abb11505fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:17.612072 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d87f2e-a348-4fff-89af-40abb11505fe admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:17.612348 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d87f2e-a348-4fff-89af-40abb11505fe admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:17.612620 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d87f2e-a348-4fff-89af-40abb11505fe admin admin] PATH_INFO: `/v3/projects/f8d8412e0594496eb913029f47537df4/users/30932c29879348bcb8396c2d9bc7e3a8/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:17.628967 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d87f2e-a348-4fff-89af-40abb11505fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8d8412e0594496eb913029f47537df4, user_id=30932c29879348bcb8396c2d9bc7e3a8, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:17.631836 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d87f2e-a348-4fff-89af-40abb11505fe admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:17.646380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 974/1940] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:17 2022] PUT /identity/v3/projects/f8d8412e0594496eb913029f47537df4/users/30932c29879348bcb8396c2d9bc7e3a8/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:17.647685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346a212-bfff-41cf-89b3-51360a795c0e None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:17.647976 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346a212-bfff-41cf-89b3-51360a795c0e None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:17.648400 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346a212-bfff-41cf-89b3-51360a795c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:17.654018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e33bd-25b4-4f4d-816f-ef6326a78bf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:17.655494 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e33bd-25b4-4f4d-816f-ef6326a78bf1 service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:17.667482 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 967/1941] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:17.668818 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbd357-c298-4e03-9701-ec2a2bc35ece None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:17.668928 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-5346a212-bfff-41cf-89b3-51360a795c0e None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:17.669173 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbd357-c298-4e03-9701-ec2a2bc35ece None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:17.669457 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbd357-c298-4e03-9701-ec2a2bc35ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:17.670187 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-5346a212-bfff-41cf-89b3-51360a795c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:17.700094 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-19bbd357-c298-4e03-9701-ec2a2bc35ece None None] MFA Rules not processed for user `30932c29879348bcb8396c2d9bc7e3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:17.701532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-19bbd357-c298-4e03-9701-ec2a2bc35ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:17.763379 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 975/1942] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:17.764265 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346a212-bfff-41cf-89b3-51360a795c0e None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:17.765629 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-5346a212-bfff-41cf-89b3-51360a795c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:17.779001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346a212-bfff-41cf-89b3-51360a795c0e None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:17.783890 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 968/1943] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:17.785330 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d90eb-f067-4679-aabf-e983f2aafab6 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:17.785616 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d90eb-f067-4679-aabf-e983f2aafab6 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:17.785892 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d90eb-f067-4679-aabf-e983f2aafab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:17.788044 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b659e3-b92e-4229-9004-f88a6a58b671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:17.788921 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b659e3-b92e-4229-9004-f88a6a58b671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:17.789234 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b659e3-b92e-4229-9004-f88a6a58b671 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:17.789509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b659e3-b92e-4229-9004-f88a6a58b671 admin admin] PATH_INFO: `/v3/users/2b35288edc8843f1907fac2197e9c0f0` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:17.790728 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b659e3-b92e-4229-9004-f88a6a58b671 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b35288edc8843f1907fac2197e9c0f0)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:17.793302 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b659e3-b92e-4229-9004-f88a6a58b671 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:17.812293 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-af5d90eb-f067-4679-aabf-e983f2aafab6 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:17.814358 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-af5d90eb-f067-4679-aabf-e983f2aafab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:17.826439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 969/1944] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:17.827479 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d90eb-f067-4679-aabf-e983f2aafab6 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:17.839405 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d90eb-f067-4679-aabf-e983f2aafab6 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:17.845047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20fd2b-aafc-4aac-ad35-d2adb223bba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:17.845740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20fd2b-aafc-4aac-ad35-d2adb223bba1 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:17.846028 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20fd2b-aafc-4aac-ad35-d2adb223bba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:17.846306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20fd2b-aafc-4aac-ad35-d2adb223bba1 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:17.847010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20fd2b-aafc-4aac-ad35-d2adb223bba1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:17.848733 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-71b659e3-b92e-4229-9004-f88a6a58b671 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b35288edc8843f1907fac2197e9c0f0'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:17.848931 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20fd2b-aafc-4aac-ad35-d2adb223bba1 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:17.857372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-71b659e3-b92e-4229-9004-f88a6a58b671 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b35288edc8843f1907fac2197e9c0f0'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:17.858027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-71b659e3-b92e-4229-9004-f88a6a58b671 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b35288edc8843f1907fac2197e9c0f0'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:17.871412 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-71b659e3-b92e-4229-9004-f88a6a58b671 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b35288edc8843f1907fac2197e9c0f0'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:17.879028 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-71b659e3-b92e-4229-9004-f88a6a58b671 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b35288edc8843f1907fac2197e9c0f0'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:17.882797 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:17.886897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 970/1945] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:17 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:17.887848 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20fd2b-aafc-4aac-ad35-d2adb223bba1 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:17.889845 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-aa20fd2b-aafc-4aac-ad35-d2adb223bba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:17.896803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 976/1946] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:17 2022] DELETE /identity/v3/users/2b35288edc8843f1907fac2197e9c0f0 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:17.902678 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b659e3-b92e-4229-9004-f88a6a58b671 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:17.905245 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-71b659e3-b92e-4229-9004-f88a6a58b671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:17.931007 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:17.994402 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b659e3-b92e-4229-9004-f88a6a58b671 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:18.000741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babea89-b357-4813-848a-2f8f21026996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:18.001376 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babea89-b357-4813-848a-2f8f21026996 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.001665 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babea89-b357-4813-848a-2f8f21026996 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.001943 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babea89-b357-4813-848a-2f8f21026996 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.002698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1babea89-b357-4813-848a-2f8f21026996 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:18.005543 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1babea89-b357-4813-848a-2f8f21026996 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:18.013524 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20fd2b-aafc-4aac-ad35-d2adb223bba1 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:18.019370 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1e6a8-9047-436a-a963-f9ef812a40d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:18.020094 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1e6a8-9047-436a-a963-f9ef812a40d1 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.020367 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1e6a8-9047-436a-a963-f9ef812a40d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.020641 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1e6a8-9047-436a-a963-f9ef812a40d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.021414 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1e6a8-9047-436a-a963-f9ef812a40d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:18.021725 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1e6a8-9047-436a-a963-f9ef812a40d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:18.023756 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1e6a8-9047-436a-a963-f9ef812a40d1 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:18.033231 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 977/1947] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:17 2022] POST /identity/v3/users => generated 298 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:18.035558 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3769e-5aa2-4539-84fc-c35f37e258ef None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.035847 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3769e-5aa2-4539-84fc-c35f37e258ef None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.036172 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3769e-5aa2-4539-84fc-c35f37e258ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.049581 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 971/1948] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:18.050418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1e6a8-9047-436a-a963-f9ef812a40d1 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:18.056000 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-0fd3769e-5aa2-4539-84fc-c35f37e258ef None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:18.057838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-0fd3769e-5aa2-4539-84fc-c35f37e258ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:18.062922 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1e6a8-9047-436a-a963-f9ef812a40d1 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:18.065992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 978/1949] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:18.067862 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ec9d4-20ce-4f3a-a58a-e35436e80757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:18.068575 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ec9d4-20ce-4f3a-a58a-e35436e80757 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.068864 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ec9d4-20ce-4f3a-a58a-e35436e80757 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.069248 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ec9d4-20ce-4f3a-a58a-e35436e80757 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.070016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ec9d4-20ce-4f3a-a58a-e35436e80757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:18.070113 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3769e-5aa2-4539-84fc-c35f37e258ef None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:18.071511 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ec9d4-20ce-4f3a-a58a-e35436e80757 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:18.081300 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3769e-5aa2-4539-84fc-c35f37e258ef None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:18.084726 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 972/1950] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:18.085580 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ec9d4-20ce-4f3a-a58a-e35436e80757 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:18.086686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3c83c-0a89-4302-abf3-d588859c4215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:18.087256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-ba5ec9d4-20ce-4f3a-a58a-e35436e80757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:18.087502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3c83c-0a89-4302-abf3-d588859c4215 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.087790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3c83c-0a89-4302-abf3-d588859c4215 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.088276 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3c83c-0a89-4302-abf3-d588859c4215 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.088900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3c83c-0a89-4302-abf3-d588859c4215 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:18.090875 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3c83c-0a89-4302-abf3-d588859c4215 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:18.123988 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 979/1951] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:18 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:18.124564 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3c83c-0a89-4302-abf3-d588859c4215 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:18.154250 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ec9d4-20ce-4f3a-a58a-e35436e80757 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:18.207244 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3c83c-0a89-4302-abf3-d588859c4215 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:18.212433 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee461b8b-5411-4120-b1ae-f30125e7fa5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:18.214087 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee461b8b-5411-4120-b1ae-f30125e7fa5a admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.214372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee461b8b-5411-4120-b1ae-f30125e7fa5a admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.214649 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee461b8b-5411-4120-b1ae-f30125e7fa5a admin admin] PATH_INFO: `/v3/projects/2f674ce8b3ec41ca821506f0c0e0b714/users/743eb9bdafb04335b25ec7f57dfee2de/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.222051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae58e52-e0aa-40f9-b790-80b8d6e25d88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:18.222734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae58e52-e0aa-40f9-b790-80b8d6e25d88 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.223102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae58e52-e0aa-40f9-b790-80b8d6e25d88 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.223383 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae58e52-e0aa-40f9-b790-80b8d6e25d88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.235002 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee461b8b-5411-4120-b1ae-f30125e7fa5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f674ce8b3ec41ca821506f0c0e0b714, user_id=743eb9bdafb04335b25ec7f57dfee2de, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:18.236424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae58e52-e0aa-40f9-b790-80b8d6e25d88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:18.237805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee461b8b-5411-4120-b1ae-f30125e7fa5a admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:18.238131 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae58e52-e0aa-40f9-b790-80b8d6e25d88 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:18.249767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 973/1952] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:18.250866 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae58e52-e0aa-40f9-b790-80b8d6e25d88 service neutron] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:18.255281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 980/1953] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:18 2022] PUT /identity/v3/projects/2f674ce8b3ec41ca821506f0c0e0b714/users/743eb9bdafb04335b25ec7f57dfee2de/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:18.256252 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee461b8b-5411-4120-b1ae-f30125e7fa5a admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:18.261104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae58e52-e0aa-40f9-b790-80b8d6e25d88 service neutron] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:18.308494 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:18.332186 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee461b8b-5411-4120-b1ae-f30125e7fa5a admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:18.336353 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3b762-23fd-451c-ab19-311ca650fa96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:18.337085 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3b762-23fd-451c-ab19-311ca650fa96 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.337374 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3b762-23fd-451c-ab19-311ca650fa96 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.337650 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3b762-23fd-451c-ab19-311ca650fa96 admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.338869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3b762-23fd-451c-ab19-311ca650fa96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:18.339290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b969f7a-d5e2-4f9d-bb27-86fa24bdd2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:18.340021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b969f7a-d5e2-4f9d-bb27-86fa24bdd2bc admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.340320 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b969f7a-d5e2-4f9d-bb27-86fa24bdd2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.340591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b969f7a-d5e2-4f9d-bb27-86fa24bdd2bc admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.340840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3b762-23fd-451c-ab19-311ca650fa96 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:18.341869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b969f7a-d5e2-4f9d-bb27-86fa24bdd2bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:18.343456 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b969f7a-d5e2-4f9d-bb27-86fa24bdd2bc admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:18.348643 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:18.357112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 981/1954] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:18.361221 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b969f7a-d5e2-4f9d-bb27-86fa24bdd2bc admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:18.370496 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 974/1955] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:18 2022] POST /identity/v3/users => generated 286 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:18.371596 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3b762-23fd-451c-ab19-311ca650fa96 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:18.383720 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b969f7a-d5e2-4f9d-bb27-86fa24bdd2bc admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:18.385133 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3b762-23fd-451c-ab19-311ca650fa96 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:18.391075 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2beb22-bab6-4b6d-a123-467d8a408e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:18.391225 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce6fb5-7ea0-4e01-ba3c-fc9f0751af0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:18.391814 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce6fb5-7ea0-4e01-ba3c-fc9f0751af0d admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.392097 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce6fb5-7ea0-4e01-ba3c-fc9f0751af0d admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.392369 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce6fb5-7ea0-4e01-ba3c-fc9f0751af0d admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.392773 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2beb22-bab6-4b6d-a123-467d8a408e3d admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.393116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce6fb5-7ea0-4e01-ba3c-fc9f0751af0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:18.393291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2beb22-bab6-4b6d-a123-467d8a408e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.393543 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2beb22-bab6-4b6d-a123-467d8a408e3d admin admin] PATH_INFO: `/v3/projects/2f674ce8b3ec41ca821506f0c0e0b714/users/743eb9bdafb04335b25ec7f57dfee2de/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.394868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce6fb5-7ea0-4e01-ba3c-fc9f0751af0d admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:18.402553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2beb22-bab6-4b6d-a123-467d8a408e3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f674ce8b3ec41ca821506f0c0e0b714, user_id=743eb9bdafb04335b25ec7f57dfee2de, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:18.402553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2beb22-bab6-4b6d-a123-467d8a408e3d admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:18.408367 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 975/1956] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:18.412945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce6fb5-7ea0-4e01-ba3c-fc9f0751af0d admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:18.424450 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce6fb5-7ea0-4e01-ba3c-fc9f0751af0d admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:18.425191 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 982/1957] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:18 2022] PUT /identity/v3/projects/2f674ce8b3ec41ca821506f0c0e0b714/users/743eb9bdafb04335b25ec7f57dfee2de/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:18.430825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8cb9a-e935-4430-93c4-c44998ff8613 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.431256 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8cb9a-e935-4430-93c4-c44998ff8613 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.431774 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8cb9a-e935-4430-93c4-c44998ff8613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.497699 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-e2d8cb9a-e935-4430-93c4-c44998ff8613 None None] MFA Rules not processed for user `743eb9bdafb04335b25ec7f57dfee2de`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:18.500437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-e2d8cb9a-e935-4430-93c4-c44998ff8613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:18.552092 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade81e61-bba4-4d69-8cfb-219246a3a135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:18.553040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade81e61-bba4-4d69-8cfb-219246a3a135 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.553369 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade81e61-bba4-4d69-8cfb-219246a3a135 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.553646 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade81e61-bba4-4d69-8cfb-219246a3a135 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.554395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade81e61-bba4-4d69-8cfb-219246a3a135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:18.557821 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade81e61-bba4-4d69-8cfb-219246a3a135 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:18.593319 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 976/1958] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:18.595321 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade81e61-bba4-4d69-8cfb-219246a3a135 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:18.606185 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 983/1959] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:18.620961 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8cb9a-e935-4430-93c4-c44998ff8613 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:18.622677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-e2d8cb9a-e935-4430-93c4-c44998ff8613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:18.639314 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade81e61-bba4-4d69-8cfb-219246a3a135 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:18.649676 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ece752-c095-4055-99e9-262b064dde4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:18.653386 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ece752-c095-4055-99e9-262b064dde4c admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.653685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ece752-c095-4055-99e9-262b064dde4c admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.653975 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ece752-c095-4055-99e9-262b064dde4c admin admin] PATH_INFO: `/v3/projects/ccffe48e478048319cc5d7ae620f8682/users/395023e7b4c44e2ba7e95b6fb3f3730a/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.688591 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ece752-c095-4055-99e9-262b064dde4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccffe48e478048319cc5d7ae620f8682, user_id=395023e7b4c44e2ba7e95b6fb3f3730a, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:18.698107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ece752-c095-4055-99e9-262b064dde4c admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:18.737592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 977/1960] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:18 2022] PUT /identity/v3/projects/ccffe48e478048319cc5d7ae620f8682/users/395023e7b4c44e2ba7e95b6fb3f3730a/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:18.739188 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ece752-c095-4055-99e9-262b064dde4c admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:18.744840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f7ece752-c095-4055-99e9-262b064dde4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:18.777502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8cb9a-e935-4430-93c4-c44998ff8613 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:18.862677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003438c-532b-438f-841c-9eca5c3f5070 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:18.863565 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003438c-532b-438f-841c-9eca5c3f5070 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.863920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003438c-532b-438f-841c-9eca5c3f5070 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.864752 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003438c-532b-438f-841c-9eca5c3f5070 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.869268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ece752-c095-4055-99e9-262b064dde4c admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:18.876335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b7312-9cab-42f2-ad8e-3028883b1732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:18.877437 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b7312-9cab-42f2-ad8e-3028883b1732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.877727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b7312-9cab-42f2-ad8e-3028883b1732 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.878646 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b7312-9cab-42f2-ad8e-3028883b1732 admin admin] PATH_INFO: `/v3/projects/c548b6a7404b426d9b72a467cd2986cc` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.880052 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b7312-9cab-42f2-ad8e-3028883b1732 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c548b6a7404b426d9b72a467cd2986cc)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:18.882069 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b7312-9cab-42f2-ad8e-3028883b1732 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:18.890697 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:18.946781 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-735b7312-9cab-42f2-ad8e-3028883b1732 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c548b6a7404b426d9b72a467cd2986cc', 'request_id': 'req-735b7312-9cab-42f2-ad8e-3028883b1732'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:18.952067 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:18.957100 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003438c-532b-438f-841c-9eca5c3f5070 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:18.959027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003438c-532b-438f-841c-9eca5c3f5070 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:18.959534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-735b7312-9cab-42f2-ad8e-3028883b1732 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c548b6a7404b426d9b72a467cd2986cc', 'request_id': 'req-735b7312-9cab-42f2-ad8e-3028883b1732'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:18.968456 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-735b7312-9cab-42f2-ad8e-3028883b1732 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c548b6a7404b426d9b72a467cd2986cc', 'request_id': 'req-735b7312-9cab-42f2-ad8e-3028883b1732'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:18.971354 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 984/1961] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:40:18.972804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d4697-ab1c-4f5a-8e31-f0be39a6ae12 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.973192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d4697-ab1c-4f5a-8e31-f0be39a6ae12 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.973474 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d4697-ab1c-4f5a-8e31-f0be39a6ae12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.982385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-735b7312-9cab-42f2-ad8e-3028883b1732 admin admin] The token cache is being invalidate because project c548b6a7404b426d9b72a467cd2986cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:18.982640 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-735b7312-9cab-42f2-ad8e-3028883b1732 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:18.983263 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-735b7312-9cab-42f2-ad8e-3028883b1732 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:18.986460 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 978/1962] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:18 2022] DELETE /identity/v3/projects/c548b6a7404b426d9b72a467cd2986cc => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:18.990401 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b7312-9cab-42f2-ad8e-3028883b1732 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:18.991838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-735b7312-9cab-42f2-ad8e-3028883b1732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:19.011529 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-b69d4697-ab1c-4f5a-8e31-f0be39a6ae12 None None] MFA Rules not processed for user `395023e7b4c44e2ba7e95b6fb3f3730a`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:19.012951 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-b69d4697-ab1c-4f5a-8e31-f0be39a6ae12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:19.096349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b7312-9cab-42f2-ad8e-3028883b1732 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:19.101970 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244760e6-3967-4f7b-9fb7-17c714de99e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:19.102895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244760e6-3967-4f7b-9fb7-17c714de99e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:19.103179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244760e6-3967-4f7b-9fb7-17c714de99e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:19.103458 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244760e6-3967-4f7b-9fb7-17c714de99e0 admin admin] PATH_INFO: `/v3/users/54bc4f3872914d7ebd1dc9a3a827c7b0` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:19.104843 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244760e6-3967-4f7b-9fb7-17c714de99e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54bc4f3872914d7ebd1dc9a3a827c7b0)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:19.106877 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244760e6-3967-4f7b-9fb7-17c714de99e0 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:19.133237 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 985/1963] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:19.135915 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d4697-ab1c-4f5a-8e31-f0be39a6ae12 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:19.137854 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-b69d4697-ab1c-4f5a-8e31-f0be39a6ae12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:19.172161 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-244760e6-3967-4f7b-9fb7-17c714de99e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54bc4f3872914d7ebd1dc9a3a827c7b0'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:19.188294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-244760e6-3967-4f7b-9fb7-17c714de99e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54bc4f3872914d7ebd1dc9a3a827c7b0'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:19.189039 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-244760e6-3967-4f7b-9fb7-17c714de99e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54bc4f3872914d7ebd1dc9a3a827c7b0'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:19.204421 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-244760e6-3967-4f7b-9fb7-17c714de99e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54bc4f3872914d7ebd1dc9a3a827c7b0'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:19.205472 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-244760e6-3967-4f7b-9fb7-17c714de99e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54bc4f3872914d7ebd1dc9a3a827c7b0'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:19.235975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d4697-ab1c-4f5a-8e31-f0be39a6ae12 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:19.239994 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0a1bc-4206-448b-aa89-eb58b8674c43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:19.240696 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0a1bc-4206-448b-aa89-eb58b8674c43 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:19.240984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0a1bc-4206-448b-aa89-eb58b8674c43 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:19.241295 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0a1bc-4206-448b-aa89-eb58b8674c43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:19.243174 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-a2b0a1bc-4206-448b-aa89-eb58b8674c43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:19.254753 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 979/1964] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:18 2022] DELETE /identity/v3/users/54bc4f3872914d7ebd1dc9a3a827c7b0 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:19.254753 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244760e6-3967-4f7b-9fb7-17c714de99e0 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:19.259523 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-244760e6-3967-4f7b-9fb7-17c714de99e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:19.357057 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0a1bc-4206-448b-aa89-eb58b8674c43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:19.360162 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0a1bc-4206-448b-aa89-eb58b8674c43 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:19.373824 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:19.381867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 986/1965] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:19.410365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244760e6-3967-4f7b-9fb7-17c714de99e0 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:19.425426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ed7ad-b190-4f37-92b7-d2f1fed6ed08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:19.426731 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ed7ad-b190-4f37-92b7-d2f1fed6ed08 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:19.427210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ed7ad-b190-4f37-92b7-d2f1fed6ed08 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:19.427733 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ed7ad-b190-4f37-92b7-d2f1fed6ed08 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:19.429243 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ed7ad-b190-4f37-92b7-d2f1fed6ed08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:19.431598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ed7ad-b190-4f37-92b7-d2f1fed6ed08 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:19.440746 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:19.494629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 980/1966] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:19 2022] POST /identity/v3/projects => generated 373 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:19.498567 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0a1bc-4206-448b-aa89-eb58b8674c43 service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:19.514503 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0a1bc-4206-448b-aa89-eb58b8674c43 service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:19.611475 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76016a-da68-4dc4-8816-0e86ef299ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:19.612760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76016a-da68-4dc4-8816-0e86ef299ede admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:19.613529 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76016a-da68-4dc4-8816-0e86ef299ede admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:19.614280 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76016a-da68-4dc4-8816-0e86ef299ede admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:19.615734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76016a-da68-4dc4-8816-0e86ef299ede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:19.618468 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76016a-da68-4dc4-8816-0e86ef299ede admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:19.658852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 987/1967] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:19 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:19.669054 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ed7ad-b190-4f37-92b7-d2f1fed6ed08 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:19.714476 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ed7ad-b190-4f37-92b7-d2f1fed6ed08 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:19.729915 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f2b61-9881-4c9d-973a-3461c0209f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:19.731968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f2b61-9881-4c9d-973a-3461c0209f8e admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:19.732263 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f2b61-9881-4c9d-973a-3461c0209f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:19.732551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f2b61-9881-4c9d-973a-3461c0209f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:19.733488 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f2b61-9881-4c9d-973a-3461c0209f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:19.737262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f2b61-9881-4c9d-973a-3461c0209f8e admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:19.774050 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 981/1968] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:19.779090 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76016a-da68-4dc4-8816-0e86ef299ede admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:19.797760 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f2b61-9881-4c9d-973a-3461c0209f8e admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:19.801114 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-6b4f2b61-9881-4c9d-973a-3461c0209f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:19.805728 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76016a-da68-4dc4-8816-0e86ef299ede admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:19.824264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1af5e5-6a85-453d-81dc-8170c36a77ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:19.825355 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1af5e5-6a85-453d-81dc-8170c36a77ed admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:19.825723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1af5e5-6a85-453d-81dc-8170c36a77ed admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:19.826127 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1af5e5-6a85-453d-81dc-8170c36a77ed admin admin] PATH_INFO: `/v3/projects/c83fd0ceea7240eb97520d53c438def6/users/5bbc4e126966492fb9dc08ba8db7034b/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:19.842162 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f2b61-9881-4c9d-973a-3461c0209f8e admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:19.851015 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475db7c3-fef0-4e73-adce-b0c268027b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:19.851937 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475db7c3-fef0-4e73-adce-b0c268027b4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:19.852218 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475db7c3-fef0-4e73-adce-b0c268027b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:19.852678 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475db7c3-fef0-4e73-adce-b0c268027b4a admin admin] PATH_INFO: `/v3/projects/57109a5ab6b741a6b734df4a2c2e1a28` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:19.856901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475db7c3-fef0-4e73-adce-b0c268027b4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=57109a5ab6b741a6b734df4a2c2e1a28)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:19.860135 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475db7c3-fef0-4e73-adce-b0c268027b4a admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:19.908867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1af5e5-6a85-453d-81dc-8170c36a77ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c83fd0ceea7240eb97520d53c438def6, user_id=5bbc4e126966492fb9dc08ba8db7034b, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:19.917323 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1af5e5-6a85-453d-81dc-8170c36a77ed admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:19.958288 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 988/1969] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:19 2022] PUT /identity/v3/projects/c83fd0ceea7240eb97520d53c438def6/users/5bbc4e126966492fb9dc08ba8db7034b/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:19.960858 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1af5e5-6a85-453d-81dc-8170c36a77ed admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:19.977634 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-475db7c3-fef0-4e73-adce-b0c268027b4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57109a5ab6b741a6b734df4a2c2e1a28', 'request_id': 'req-475db7c3-fef0-4e73-adce-b0c268027b4a'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:19.993651 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-475db7c3-fef0-4e73-adce-b0c268027b4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57109a5ab6b741a6b734df4a2c2e1a28', 'request_id': 'req-475db7c3-fef0-4e73-adce-b0c268027b4a'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:20.009983 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-475db7c3-fef0-4e73-adce-b0c268027b4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57109a5ab6b741a6b734df4a2c2e1a28', 'request_id': 'req-475db7c3-fef0-4e73-adce-b0c268027b4a'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:20.037642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-475db7c3-fef0-4e73-adce-b0c268027b4a admin admin] The token cache is being invalidate because project 57109a5ab6b741a6b734df4a2c2e1a28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:20.037888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-475db7c3-fef0-4e73-adce-b0c268027b4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:20.048017 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-475db7c3-fef0-4e73-adce-b0c268027b4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:20.056962 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 982/1970] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:19 2022] DELETE /identity/v3/projects/57109a5ab6b741a6b734df4a2c2e1a28 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:20.062943 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f137bc-9141-4e44-bebb-93d4b289dc8d None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:20.063234 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f137bc-9141-4e44-bebb-93d4b289dc8d None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:20.063512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f137bc-9141-4e44-bebb-93d4b289dc8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:20.134687 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-77f137bc-9141-4e44-bebb-93d4b289dc8d None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:20.135977 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-77f137bc-9141-4e44-bebb-93d4b289dc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:20.246393 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1af5e5-6a85-453d-81dc-8170c36a77ed admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:20.332698 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 983/1971] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:20.341124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bb5fe-e515-45f5-af57-c2d29e5a2613 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:20.342620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bb5fe-e515-45f5-af57-c2d29e5a2613 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:20.343587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bb5fe-e515-45f5-af57-c2d29e5a2613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:20.366854 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d44939-27ac-4f98-a891-20ff7310d22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:20.367751 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d44939-27ac-4f98-a891-20ff7310d22a admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:20.368097 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d44939-27ac-4f98-a891-20ff7310d22a admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:20.368532 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d44939-27ac-4f98-a891-20ff7310d22a admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:20.371046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d44939-27ac-4f98-a891-20ff7310d22a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:20.375891 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d44939-27ac-4f98-a891-20ff7310d22a admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:20.386168 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-f27bb5fe-e515-45f5-af57-c2d29e5a2613 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:20.390248 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f27bb5fe-e515-45f5-af57-c2d29e5a2613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:20.411316 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 989/1972] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:20.413704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d44939-27ac-4f98-a891-20ff7310d22a admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:20.418302 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-91d44939-27ac-4f98-a891-20ff7310d22a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:20.418888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 984/1973] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:20.422278 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bb5fe-e515-45f5-af57-c2d29e5a2613 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:20.439406 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bb5fe-e515-45f5-af57-c2d29e5a2613 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:20.445201 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d44939-27ac-4f98-a891-20ff7310d22a admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:20.461580 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d959f-de01-4635-8c72-1e9ebe906067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:20.464121 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d959f-de01-4635-8c72-1e9ebe906067 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:20.464726 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d959f-de01-4635-8c72-1e9ebe906067 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:20.466548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d959f-de01-4635-8c72-1e9ebe906067 admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:20.468439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d959f-de01-4635-8c72-1e9ebe906067 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:20.469213 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d959f-de01-4635-8c72-1e9ebe906067 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:20.470934 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397a28c-7f6a-442c-b7a6-d2ec055a7b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:20.471107 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397a28c-7f6a-442c-b7a6-d2ec055a7b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:20.471303 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397a28c-7f6a-442c-b7a6-d2ec055a7b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:20.474650 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d959f-de01-4635-8c72-1e9ebe906067 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:20.474947 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397a28c-7f6a-442c-b7a6-d2ec055a7b8e admin admin] PATH_INFO: `/v3/projects/c83fd0ceea7240eb97520d53c438def6/users/5bbc4e126966492fb9dc08ba8db7034b/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:20.479394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397a28c-7f6a-442c-b7a6-d2ec055a7b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c83fd0ceea7240eb97520d53c438def6, user_id=5bbc4e126966492fb9dc08ba8db7034b, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:20.489305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397a28c-7f6a-442c-b7a6-d2ec055a7b8e admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:20.493689 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:20.512790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 985/1974] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:20.517646 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d959f-de01-4635-8c72-1e9ebe906067 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:20.537095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 990/1975] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:20 2022] PUT /identity/v3/projects/c83fd0ceea7240eb97520d53c438def6/users/5bbc4e126966492fb9dc08ba8db7034b/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:20.539863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f734af-7970-44bb-8fdb-38fff2e94fdf None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:20.540365 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f734af-7970-44bb-8fdb-38fff2e94fdf None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:20.540910 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f734af-7970-44bb-8fdb-38fff2e94fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:20.543353 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d959f-de01-4635-8c72-1e9ebe906067 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:20.578284 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-02f734af-7970-44bb-8fdb-38fff2e94fdf None None] MFA Rules not processed for user `5bbc4e126966492fb9dc08ba8db7034b`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:20.580285 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-02f734af-7970-44bb-8fdb-38fff2e94fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:20.592965 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:20.626378 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63325b4-2716-42b5-89bd-aedcc9cef495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:20.627908 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63325b4-2716-42b5-89bd-aedcc9cef495 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:20.628752 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63325b4-2716-42b5-89bd-aedcc9cef495 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:20.629230 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63325b4-2716-42b5-89bd-aedcc9cef495 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:20.631063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63325b4-2716-42b5-89bd-aedcc9cef495 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:20.633494 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63325b4-2716-42b5-89bd-aedcc9cef495 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:20.671847 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 986/1976] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:20 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:40:20.672801 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63325b4-2716-42b5-89bd-aedcc9cef495 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:20.680804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 991/1977] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:20.693677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f734af-7970-44bb-8fdb-38fff2e94fdf None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:20.696985 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-02f734af-7970-44bb-8fdb-38fff2e94fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:20.758535 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63325b4-2716-42b5-89bd-aedcc9cef495 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:20.764996 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c0307-1294-432f-8168-8c0c14d46ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:20.765411 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c0307-1294-432f-8168-8c0c14d46ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:20.765785 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c0307-1294-432f-8168-8c0c14d46ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:20.766253 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c0307-1294-432f-8168-8c0c14d46ad8 admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:20.767028 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c0307-1294-432f-8168-8c0c14d46ad8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:20.769881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c0307-1294-432f-8168-8c0c14d46ad8 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:20.790982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f734af-7970-44bb-8fdb-38fff2e94fdf None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:20.796051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 987/1978] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:20 2022] POST /identity/v3/users => generated 290 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:20.800584 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c0307-1294-432f-8168-8c0c14d46ad8 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:20.812838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c0307-1294-432f-8168-8c0c14d46ad8 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:20.818119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8fe1d-721e-4ae2-9be1-2710102b46fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:20.818814 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8fe1d-721e-4ae2-9be1-2710102b46fa admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:20.819092 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8fe1d-721e-4ae2-9be1-2710102b46fa admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:20.819372 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8fe1d-721e-4ae2-9be1-2710102b46fa admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:20.820214 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8fe1d-721e-4ae2-9be1-2710102b46fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:20.821973 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8fe1d-721e-4ae2-9be1-2710102b46fa admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:20.837789 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 988/1979] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:40:20.839166 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8fe1d-721e-4ae2-9be1-2710102b46fa admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:20.852405 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8fe1d-721e-4ae2-9be1-2710102b46fa admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:20.859557 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da753731-a832-4a9f-8c79-02fc2cf83e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:20.860472 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da753731-a832-4a9f-8c79-02fc2cf83e8f admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:20.860759 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da753731-a832-4a9f-8c79-02fc2cf83e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:20.861034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da753731-a832-4a9f-8c79-02fc2cf83e8f admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:20.861826 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da753731-a832-4a9f-8c79-02fc2cf83e8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:20.863563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da753731-a832-4a9f-8c79-02fc2cf83e8f admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:20.864068 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f23b16-8ea5-4a06-ac66-1309008a9a41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:20.865014 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f23b16-8ea5-4a06-ac66-1309008a9a41 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:20.865538 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f23b16-8ea5-4a06-ac66-1309008a9a41 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:20.865889 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f23b16-8ea5-4a06-ac66-1309008a9a41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:20.868044 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-a9f23b16-8ea5-4a06-ac66-1309008a9a41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:20.880828 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 989/1980] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:20.890884 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da753731-a832-4a9f-8c79-02fc2cf83e8f admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:20.903274 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da753731-a832-4a9f-8c79-02fc2cf83e8f admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:20.918953 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707914c-4133-40b7-9e35-eeead0fe843c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:20.919985 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707914c-4133-40b7-9e35-eeead0fe843c admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:20.920267 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707914c-4133-40b7-9e35-eeead0fe843c admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:20.920542 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707914c-4133-40b7-9e35-eeead0fe843c admin admin] PATH_INFO: `/v3/projects/bf05f15289d64295831495883933b569/users/52308f4edb994261a40dcb494cb512cc/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:20.940531 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707914c-4133-40b7-9e35-eeead0fe843c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf05f15289d64295831495883933b569, user_id=52308f4edb994261a40dcb494cb512cc, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:20.943315 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707914c-4133-40b7-9e35-eeead0fe843c admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:20.943669 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:20.956568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f23b16-8ea5-4a06-ac66-1309008a9a41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:20.958170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f23b16-8ea5-4a06-ac66-1309008a9a41 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:20.958829 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 990/1981] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:20 2022] PUT /identity/v3/projects/bf05f15289d64295831495883933b569/users/52308f4edb994261a40dcb494cb512cc/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:20.969891 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d768b9d-86a4-41fd-a6c3-c0505925ae65 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:20.970227 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d768b9d-86a4-41fd-a6c3-c0505925ae65 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:20.970443 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d768b9d-86a4-41fd-a6c3-c0505925ae65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:20.975493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 992/1982] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:40:21.003923 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-0d768b9d-86a4-41fd-a6c3-c0505925ae65 None None] MFA Rules not processed for user `52308f4edb994261a40dcb494cb512cc`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:21.005418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-0d768b9d-86a4-41fd-a6c3-c0505925ae65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:21.075707 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:21.088983 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 991/1983] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:21.094482 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f23b16-8ea5-4a06-ac66-1309008a9a41 service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:21.108376 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f23b16-8ea5-4a06-ac66-1309008a9a41 service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:21.218933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e168c-8954-49a4-a692-a121b3a523b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:21.220561 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e168c-8954-49a4-a692-a121b3a523b4 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:21.221292 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e168c-8954-49a4-a692-a121b3a523b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:21.221980 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e168c-8954-49a4-a692-a121b3a523b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:21.223052 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e168c-8954-49a4-a692-a121b3a523b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:21.225994 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e168c-8954-49a4-a692-a121b3a523b4 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:21.271903 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 993/1984] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:21 2022] POST /identity/v3/projects => generated 381 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:21.276840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d768b9d-86a4-41fd-a6c3-c0505925ae65 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:21.300908 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d768b9d-86a4-41fd-a6c3-c0505925ae65 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:21.443539 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7223ba-08d0-46a3-a98f-63a5db37d184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:21.451351 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7223ba-08d0-46a3-a98f-63a5db37d184 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:21.451820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7223ba-08d0-46a3-a98f-63a5db37d184 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:21.452404 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7223ba-08d0-46a3-a98f-63a5db37d184 admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:21.453611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7223ba-08d0-46a3-a98f-63a5db37d184 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:21.456166 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7223ba-08d0-46a3-a98f-63a5db37d184 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:21.489771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 992/1985] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:21 2022] POST /identity/v3/users => generated 297 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:21.498870 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e168c-8954-49a4-a692-a121b3a523b4 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:21.506011 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e168c-8954-49a4-a692-a121b3a523b4 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:21.511314 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6569a9-3895-412c-9587-25d2e3998b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:21.511986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6569a9-3895-412c-9587-25d2e3998b04 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:21.512272 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6569a9-3895-412c-9587-25d2e3998b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:21.512552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6569a9-3895-412c-9587-25d2e3998b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:21.513333 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6569a9-3895-412c-9587-25d2e3998b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:21.514956 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6569a9-3895-412c-9587-25d2e3998b04 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:21.526793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 994/1986] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:21.531238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7223ba-08d0-46a3-a98f-63a5db37d184 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:21.543020 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7223ba-08d0-46a3-a98f-63a5db37d184 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:21.550335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47eecc-cc54-4147-80ab-afc974aa8ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:21.551337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47eecc-cc54-4147-80ab-afc974aa8ad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:21.551800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47eecc-cc54-4147-80ab-afc974aa8ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:21.552150 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47eecc-cc54-4147-80ab-afc974aa8ad9 admin admin] PATH_INFO: `/v3/projects/d57592da1c594c91a05833d7c13e45eb/users/e06d2829c05e4676b4cbef9ecdb2d678/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:21.575654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47eecc-cc54-4147-80ab-afc974aa8ad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d57592da1c594c91a05833d7c13e45eb, user_id=e06d2829c05e4676b4cbef9ecdb2d678, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:21.578574 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47eecc-cc54-4147-80ab-afc974aa8ad9 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:21.592874 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 993/1987] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:21 2022] PUT /identity/v3/projects/d57592da1c594c91a05833d7c13e45eb/users/e06d2829c05e4676b4cbef9ecdb2d678/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:21.597215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6569a9-3895-412c-9587-25d2e3998b04 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:21.608502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6569a9-3895-412c-9587-25d2e3998b04 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:21.668844 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b23ac5-2972-4acc-9de6-ac611d7eb152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:21.669548 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b23ac5-2972-4acc-9de6-ac611d7eb152 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:21.669857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b23ac5-2972-4acc-9de6-ac611d7eb152 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:21.670137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b23ac5-2972-4acc-9de6-ac611d7eb152 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:21.670888 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b23ac5-2972-4acc-9de6-ac611d7eb152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:21.672480 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b23ac5-2972-4acc-9de6-ac611d7eb152 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:21.683409 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 995/1988] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:40:21.687790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47eecc-cc54-4147-80ab-afc974aa8ad9 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:21.700849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47eecc-cc54-4147-80ab-afc974aa8ad9 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:21.705725 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34684891-b61d-449d-86d4-5f366ff2b05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:21.706651 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34684891-b61d-449d-86d4-5f366ff2b05f admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:21.706929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34684891-b61d-449d-86d4-5f366ff2b05f admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:21.707201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34684891-b61d-449d-86d4-5f366ff2b05f admin admin] PATH_INFO: `/v3/projects/d57592da1c594c91a05833d7c13e45eb/users/e06d2829c05e4676b4cbef9ecdb2d678/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:21.709353 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34684891-b61d-449d-86d4-5f366ff2b05f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d57592da1c594c91a05833d7c13e45eb, user_id=e06d2829c05e4676b4cbef9ecdb2d678, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:21.712126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34684891-b61d-449d-86d4-5f366ff2b05f admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:21.723954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 994/1989] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:21 2022] PUT /identity/v3/projects/d57592da1c594c91a05833d7c13e45eb/users/e06d2829c05e4676b4cbef9ecdb2d678/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:21.736535 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735a98a-82d1-4cbb-aefe-691242a9c0d2 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:21.736944 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735a98a-82d1-4cbb-aefe-691242a9c0d2 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:21.737491 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735a98a-82d1-4cbb-aefe-691242a9c0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:21.777907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-f735a98a-82d1-4cbb-aefe-691242a9c0d2 None None] MFA Rules not processed for user `e06d2829c05e4676b4cbef9ecdb2d678`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:21.779145 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-f735a98a-82d1-4cbb-aefe-691242a9c0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:21.843327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:21.869247 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 996/1990] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:21.882108 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34684891-b61d-449d-86d4-5f366ff2b05f admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:21.944918 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34684891-b61d-449d-86d4-5f366ff2b05f admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:21.950646 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09183e-b18d-4313-810f-a73257282963 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:21.951473 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09183e-b18d-4313-810f-a73257282963 service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:21.951829 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09183e-b18d-4313-810f-a73257282963 service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:21.952179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09183e-b18d-4313-810f-a73257282963 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:21.962606 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09183e-b18d-4313-810f-a73257282963 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:21.964939 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09183e-b18d-4313-810f-a73257282963 service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:21.977052 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 995/1991] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:22.325222 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735a98a-82d1-4cbb-aefe-691242a9c0d2 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:22.335112 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735a98a-82d1-4cbb-aefe-691242a9c0d2 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:22.340765 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e204f2-be87-4461-b089-d6c94e7e9385 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06d2829c05e4676b4cbef9ecdb2d678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd57592da1c594c91a05833d7c13e45eb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:22.341532 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e204f2-be87-4461-b089-d6c94e7e9385 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:22.341820 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e204f2-be87-4461-b089-d6c94e7e9385 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:22.342095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e204f2-be87-4461-b089-d6c94e7e9385 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:22.342835 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e204f2-be87-4461-b089-d6c94e7e9385 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:22.343148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e204f2-be87-4461-b089-d6c94e7e9385 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:22.345201 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e204f2-be87-4461-b089-d6c94e7e9385 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:22.362838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 997/1992] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:22.363532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09183e-b18d-4313-810f-a73257282963 service cinder] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:22.376373 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09183e-b18d-4313-810f-a73257282963 service cinder] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:22.389592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896f0fe-47e8-4126-b95b-c775d116e6d6 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06d2829c05e4676b4cbef9ecdb2d678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd57592da1c594c91a05833d7c13e45eb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:22.390370 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896f0fe-47e8-4126-b95b-c775d116e6d6 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:22.390733 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896f0fe-47e8-4126-b95b-c775d116e6d6 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:22.391093 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896f0fe-47e8-4126-b95b-c775d116e6d6 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:22.391866 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896f0fe-47e8-4126-b95b-c775d116e6d6 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:22.393945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896f0fe-47e8-4126-b95b-c775d116e6d6 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:22.435867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 996/1993] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:22 2022] POST /identity/v3/projects => generated 355 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:40:22.437369 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e204f2-be87-4461-b089-d6c94e7e9385 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:22.448049 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e204f2-be87-4461-b089-d6c94e7e9385 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:22.520350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636f4e6-e846-431e-ab48-a18a0c7509b6 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06d2829c05e4676b4cbef9ecdb2d678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd57592da1c594c91a05833d7c13e45eb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:22.521496 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636f4e6-e846-431e-ab48-a18a0c7509b6 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:22.521932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636f4e6-e846-431e-ab48-a18a0c7509b6 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:22.522368 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636f4e6-e846-431e-ab48-a18a0c7509b6 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:22.523254 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636f4e6-e846-431e-ab48-a18a0c7509b6 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:22.523729 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636f4e6-e846-431e-ab48-a18a0c7509b6 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:22.525771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636f4e6-e846-431e-ab48-a18a0c7509b6 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:22.542529 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 998/1994] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:22.584097 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896f0fe-47e8-4126-b95b-c775d116e6d6 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:22.594500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896f0fe-47e8-4126-b95b-c775d116e6d6 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:22.603331 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aade2a7-867d-4059-ac9f-18d2963c46b2 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06d2829c05e4676b4cbef9ecdb2d678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd57592da1c594c91a05833d7c13e45eb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:22.604807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aade2a7-867d-4059-ac9f-18d2963c46b2 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:22.605183 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aade2a7-867d-4059-ac9f-18d2963c46b2 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:22.605526 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aade2a7-867d-4059-ac9f-18d2963c46b2 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] PATH_INFO: `/v3/projects/d95862f20c6c43b4945bffe51bfebd1e` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:22.606794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aade2a7-867d-4059-ac9f-18d2963c46b2 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d95862f20c6c43b4945bffe51bfebd1e)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:22.608795 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aade2a7-867d-4059-ac9f-18d2963c46b2 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:22.634221 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:22.660786 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-7aade2a7-867d-4059-ac9f-18d2963c46b2 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd95862f20c6c43b4945bffe51bfebd1e', 'request_id': 'req-7aade2a7-867d-4059-ac9f-18d2963c46b2'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:22.668911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-7aade2a7-867d-4059-ac9f-18d2963c46b2 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd95862f20c6c43b4945bffe51bfebd1e', 'request_id': 'req-7aade2a7-867d-4059-ac9f-18d2963c46b2'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:22.675132 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-7aade2a7-867d-4059-ac9f-18d2963c46b2 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd95862f20c6c43b4945bffe51bfebd1e', 'request_id': 'req-7aade2a7-867d-4059-ac9f-18d2963c46b2'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:22.687353 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-7aade2a7-867d-4059-ac9f-18d2963c46b2 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] The token cache is being invalidate because project d95862f20c6c43b4945bffe51bfebd1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:22.687647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-7aade2a7-867d-4059-ac9f-18d2963c46b2 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:22.688362 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-7aade2a7-867d-4059-ac9f-18d2963c46b2 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:22.689971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 997/1995] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:22 2022] DELETE /identity/v3/projects/d95862f20c6c43b4945bffe51bfebd1e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:22.952345 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636f4e6-e846-431e-ab48-a18a0c7509b6 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:22.954807 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-3636f4e6-e846-431e-ab48-a18a0c7509b6 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:23.047362 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636f4e6-e846-431e-ab48-a18a0c7509b6 tempest-VolumeQuotaClassesTest-1157628856 tempest-VolumeQuotaClassesTest-1157628856-project-admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:23.127970 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b67ff-35a4-48c2-ba71-6e8e729d30cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:23.129185 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b67ff-35a4-48c2-ba71-6e8e729d30cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:23.129639 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b67ff-35a4-48c2-ba71-6e8e729d30cd admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:23.130069 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b67ff-35a4-48c2-ba71-6e8e729d30cd admin admin] PATH_INFO: `/v3/users/52308f4edb994261a40dcb494cb512cc` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:23.132492 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b67ff-35a4-48c2-ba71-6e8e729d30cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=52308f4edb994261a40dcb494cb512cc)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:23.134854 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b67ff-35a4-48c2-ba71-6e8e729d30cd admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:23.153609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:23.195522 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4a6b67ff-35a4-48c2-ba71-6e8e729d30cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52308f4edb994261a40dcb494cb512cc'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:23.205001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4a6b67ff-35a4-48c2-ba71-6e8e729d30cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52308f4edb994261a40dcb494cb512cc'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:23.206059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4a6b67ff-35a4-48c2-ba71-6e8e729d30cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52308f4edb994261a40dcb494cb512cc'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:23.222731 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4a6b67ff-35a4-48c2-ba71-6e8e729d30cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52308f4edb994261a40dcb494cb512cc'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:23.233059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4a6b67ff-35a4-48c2-ba71-6e8e729d30cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52308f4edb994261a40dcb494cb512cc'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:23.240688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 999/1996] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:22 2022] DELETE /identity/v3/users/52308f4edb994261a40dcb494cb512cc => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:23.246058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263aa63e-6371-434a-bf41-0e1e0cb17bef None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:23.246365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263aa63e-6371-434a-bf41-0e1e0cb17bef None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:23.246694 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263aa63e-6371-434a-bf41-0e1e0cb17bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:23.266950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-263aa63e-6371-434a-bf41-0e1e0cb17bef None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:23.268208 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-263aa63e-6371-434a-bf41-0e1e0cb17bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:23.337411 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 998/1997] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:23.352370 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b67ff-35a4-48c2-ba71-6e8e729d30cd admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:23.355499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-4a6b67ff-35a4-48c2-ba71-6e8e729d30cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:23.431463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b67ff-35a4-48c2-ba71-6e8e729d30cd admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:23.435845 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d4887-543f-49a2-8e82-158ddfd3bb9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:23.436613 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d4887-543f-49a2-8e82-158ddfd3bb9a service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:23.436961 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d4887-543f-49a2-8e82-158ddfd3bb9a service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:23.437366 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d4887-543f-49a2-8e82-158ddfd3bb9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:23.449318 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d4887-543f-49a2-8e82-158ddfd3bb9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:23.457934 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d4887-543f-49a2-8e82-158ddfd3bb9a service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:23.468999 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1000/1998] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:23.770849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263aa63e-6371-434a-bf41-0e1e0cb17bef None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:23.772588 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-263aa63e-6371-434a-bf41-0e1e0cb17bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:23.786858 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263aa63e-6371-434a-bf41-0e1e0cb17bef None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:23.792258 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3aeb79-cc2e-4af6-bf9b-95bde3ca20ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:23.793320 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3aeb79-cc2e-4af6-bf9b-95bde3ca20ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:23.793605 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3aeb79-cc2e-4af6-bf9b-95bde3ca20ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:23.793882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3aeb79-cc2e-4af6-bf9b-95bde3ca20ca admin admin] PATH_INFO: `/v3/projects/bf05f15289d64295831495883933b569` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:23.796198 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3aeb79-cc2e-4af6-bf9b-95bde3ca20ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf05f15289d64295831495883933b569)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:23.798453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3aeb79-cc2e-4af6-bf9b-95bde3ca20ca admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:23.846779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-0b3aeb79-cc2e-4af6-bf9b-95bde3ca20ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf05f15289d64295831495883933b569', 'request_id': 'req-0b3aeb79-cc2e-4af6-bf9b-95bde3ca20ca'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:23.853831 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-0b3aeb79-cc2e-4af6-bf9b-95bde3ca20ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf05f15289d64295831495883933b569', 'request_id': 'req-0b3aeb79-cc2e-4af6-bf9b-95bde3ca20ca'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:23.859703 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-0b3aeb79-cc2e-4af6-bf9b-95bde3ca20ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf05f15289d64295831495883933b569', 'request_id': 'req-0b3aeb79-cc2e-4af6-bf9b-95bde3ca20ca'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:23.870663 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-0b3aeb79-cc2e-4af6-bf9b-95bde3ca20ca admin admin] The token cache is being invalidate because project bf05f15289d64295831495883933b569 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:23.870947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-0b3aeb79-cc2e-4af6-bf9b-95bde3ca20ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:23.871637 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-0b3aeb79-cc2e-4af6-bf9b-95bde3ca20ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:23.873195 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 999/1999] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:23 2022] DELETE /identity/v3/projects/bf05f15289d64295831495883933b569 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:23.877357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d4887-543f-49a2-8e82-158ddfd3bb9a service neutron] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:23.887583 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d4887-543f-49a2-8e82-158ddfd3bb9a service neutron] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:23.944313 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902dd62-f618-47e2-81dc-54895de08ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:23.945236 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902dd62-f618-47e2-81dc-54895de08ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:23.945522 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902dd62-f618-47e2-81dc-54895de08ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:23.945796 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902dd62-f618-47e2-81dc-54895de08ed6 admin admin] PATH_INFO: `/v3/users/e06d2829c05e4676b4cbef9ecdb2d678` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:23.947031 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902dd62-f618-47e2-81dc-54895de08ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e06d2829c05e4676b4cbef9ecdb2d678)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:23.949498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902dd62-f618-47e2-81dc-54895de08ed6 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:24.000098 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:24.007951 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-b902dd62-f618-47e2-81dc-54895de08ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e06d2829c05e4676b4cbef9ecdb2d678'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:24.016917 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-b902dd62-f618-47e2-81dc-54895de08ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e06d2829c05e4676b4cbef9ecdb2d678'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:24.017762 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-b902dd62-f618-47e2-81dc-54895de08ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e06d2829c05e4676b4cbef9ecdb2d678'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:24.032442 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-b902dd62-f618-47e2-81dc-54895de08ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e06d2829c05e4676b4cbef9ecdb2d678'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:24.039970 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-b902dd62-f618-47e2-81dc-54895de08ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e06d2829c05e4676b4cbef9ecdb2d678'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:24.046098 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1001/2000] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:23 2022] DELETE /identity/v3/users/e06d2829c05e4676b4cbef9ecdb2d678 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:24.348939 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3aeb79-cc2e-4af6-bf9b-95bde3ca20ca admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:24.350446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-0b3aeb79-cc2e-4af6-bf9b-95bde3ca20ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:24.365199 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3aeb79-cc2e-4af6-bf9b-95bde3ca20ca admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:24.402188 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:24.430464 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45ef77-3d2e-4e6e-85b7-e3442954ae2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:24.431326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45ef77-3d2e-4e6e-85b7-e3442954ae2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:24.431608 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45ef77-3d2e-4e6e-85b7-e3442954ae2a admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:24.432391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45ef77-3d2e-4e6e-85b7-e3442954ae2a admin admin] PATH_INFO: `/v3/projects/d57592da1c594c91a05833d7c13e45eb` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:24.433713 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45ef77-3d2e-4e6e-85b7-e3442954ae2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d57592da1c594c91a05833d7c13e45eb)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:24.435640 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45ef77-3d2e-4e6e-85b7-e3442954ae2a admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:24.488120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-6c45ef77-3d2e-4e6e-85b7-e3442954ae2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd57592da1c594c91a05833d7c13e45eb', 'request_id': 'req-6c45ef77-3d2e-4e6e-85b7-e3442954ae2a'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:24.501817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-6c45ef77-3d2e-4e6e-85b7-e3442954ae2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd57592da1c594c91a05833d7c13e45eb', 'request_id': 'req-6c45ef77-3d2e-4e6e-85b7-e3442954ae2a'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:24.521511 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-6c45ef77-3d2e-4e6e-85b7-e3442954ae2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd57592da1c594c91a05833d7c13e45eb', 'request_id': 'req-6c45ef77-3d2e-4e6e-85b7-e3442954ae2a'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:24.540928 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-6c45ef77-3d2e-4e6e-85b7-e3442954ae2a admin admin] The token cache is being invalidate because project d57592da1c594c91a05833d7c13e45eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:24.541290 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-6c45ef77-3d2e-4e6e-85b7-e3442954ae2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:24.550029 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-6c45ef77-3d2e-4e6e-85b7-e3442954ae2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:24.552485 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1000/2001] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:24 2022] DELETE /identity/v3/projects/d57592da1c594c91a05833d7c13e45eb => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:24.557013 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650515b-0bc0-4e4e-b339-c57cc1adfd54 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:24.557535 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650515b-0bc0-4e4e-b339-c57cc1adfd54 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:24.557638 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650515b-0bc0-4e4e-b339-c57cc1adfd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:24.579478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-2650515b-0bc0-4e4e-b339-c57cc1adfd54 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:24.580711 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-2650515b-0bc0-4e4e-b339-c57cc1adfd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:24.664986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45ef77-3d2e-4e6e-85b7-e3442954ae2a admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:24.667346 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-6c45ef77-3d2e-4e6e-85b7-e3442954ae2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:24.716902 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1002/2002] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:24.727297 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75849c84-926f-4a41-81fe-6acc53dc6254 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:24.727635 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75849c84-926f-4a41-81fe-6acc53dc6254 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:24.728048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75849c84-926f-4a41-81fe-6acc53dc6254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:24.757726 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-75849c84-926f-4a41-81fe-6acc53dc6254 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:24.760704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-75849c84-926f-4a41-81fe-6acc53dc6254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:24.773978 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1003/2003] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:24.782819 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75849c84-926f-4a41-81fe-6acc53dc6254 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:24.798819 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45ef77-3d2e-4e6e-85b7-e3442954ae2a admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:24.804665 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75849c84-926f-4a41-81fe-6acc53dc6254 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:24.804840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb51204-9950-4308-a6b9-508a6c649c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:24.808276 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb51204-9950-4308-a6b9-508a6c649c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:24.809601 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb51204-9950-4308-a6b9-508a6c649c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:24.810536 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574a830-0aa0-4ac7-bd7c-a906bec42008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:24.811374 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574a830-0aa0-4ac7-bd7c-a906bec42008 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:24.811684 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574a830-0aa0-4ac7-bd7c-a906bec42008 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:24.811970 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574a830-0aa0-4ac7-bd7c-a906bec42008 admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:24.812603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb51204-9950-4308-a6b9-508a6c649c95 admin admin] PATH_INFO: `/v3/users/30932c29879348bcb8396c2d9bc7e3a8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:24.813023 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0574a830-0aa0-4ac7-bd7c-a906bec42008 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:24.814426 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0574a830-0aa0-4ac7-bd7c-a906bec42008 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:24.814561 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb51204-9950-4308-a6b9-508a6c649c95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30932c29879348bcb8396c2d9bc7e3a8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:24.815807 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0574a830-0aa0-4ac7-bd7c-a906bec42008 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:24.821468 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb51204-9950-4308-a6b9-508a6c649c95 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:24.833787 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1004/2004] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:24.841804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574a830-0aa0-4ac7-bd7c-a906bec42008 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:24.874055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574a830-0aa0-4ac7-bd7c-a906bec42008 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:24.887213 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5d632-b4ba-4e95-82f0-66f4d494d0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:24.890900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5d632-b4ba-4e95-82f0-66f4d494d0e2 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:24.891281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5d632-b4ba-4e95-82f0-66f4d494d0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:24.891635 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5d632-b4ba-4e95-82f0-66f4d494d0e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:24.892468 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5d632-b4ba-4e95-82f0-66f4d494d0e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:24.894902 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5d632-b4ba-4e95-82f0-66f4d494d0e2 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:24.946061 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-5eb51204-9950-4308-a6b9-508a6c649c95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30932c29879348bcb8396c2d9bc7e3a8'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:24.963789 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-5eb51204-9950-4308-a6b9-508a6c649c95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30932c29879348bcb8396c2d9bc7e3a8'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:24.969109 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-5eb51204-9950-4308-a6b9-508a6c649c95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30932c29879348bcb8396c2d9bc7e3a8'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:24.983733 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1005/2005] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:24 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:24.987546 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-5eb51204-9950-4308-a6b9-508a6c649c95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30932c29879348bcb8396c2d9bc7e3a8'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:24.988800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5d632-b4ba-4e95-82f0-66f4d494d0e2 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:24.989593 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-5eb51204-9950-4308-a6b9-508a6c649c95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30932c29879348bcb8396c2d9bc7e3a8'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:24.992125 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d5d5d632-b4ba-4e95-82f0-66f4d494d0e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:25.000636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:25.031643 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1001/2006] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:24 2022] DELETE /identity/v3/users/30932c29879348bcb8396c2d9bc7e3a8 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:25.036092 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2d38b-0904-46bf-a2a2-ee372dd3a8be None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:25.036447 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2d38b-0904-46bf-a2a2-ee372dd3a8be None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:25.036838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2d38b-0904-46bf-a2a2-ee372dd3a8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:25.065945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-16e2d38b-0904-46bf-a2a2-ee372dd3a8be None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:25.067515 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-16e2d38b-0904-46bf-a2a2-ee372dd3a8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:25.101864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5d632-b4ba-4e95-82f0-66f4d494d0e2 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:25.123121 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:25.143104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1002/2007] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:25.154617 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2d38b-0904-46bf-a2a2-ee372dd3a8be None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:25.156085 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-16e2d38b-0904-46bf-a2a2-ee372dd3a8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:25.167575 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e45fd-6007-45e1-a128-49c192a18972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:25.168174 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e45fd-6007-45e1-a128-49c192a18972 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:25.168458 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e45fd-6007-45e1-a128-49c192a18972 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:25.168741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e45fd-6007-45e1-a128-49c192a18972 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:25.169484 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e45fd-6007-45e1-a128-49c192a18972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:25.171890 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e45fd-6007-45e1-a128-49c192a18972 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:25.194102 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1006/2008] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:24 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:40:25.198474 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e45fd-6007-45e1-a128-49c192a18972 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:25.210618 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e45fd-6007-45e1-a128-49c192a18972 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:25.216734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f1890-2e37-44fa-80a1-af9c927d5f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:25.217353 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f1890-2e37-44fa-80a1-af9c927d5f60 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:25.217639 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f1890-2e37-44fa-80a1-af9c927d5f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:25.217973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f1890-2e37-44fa-80a1-af9c927d5f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:25.218896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f1890-2e37-44fa-80a1-af9c927d5f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:25.220864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f1890-2e37-44fa-80a1-af9c927d5f60 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:25.225590 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2d38b-0904-46bf-a2a2-ee372dd3a8be None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:25.230179 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c46a0-7866-414e-b35d-01d0ec8bfe4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:25.230846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c46a0-7866-414e-b35d-01d0ec8bfe4a service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:25.231168 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c46a0-7866-414e-b35d-01d0ec8bfe4a service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:25.231511 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c46a0-7866-414e-b35d-01d0ec8bfe4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:25.233200 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1007/2009] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:25.236529 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f1890-2e37-44fa-80a1-af9c927d5f60 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:25.243626 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c46a0-7866-414e-b35d-01d0ec8bfe4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:25.245255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c46a0-7866-414e-b35d-01d0ec8bfe4a service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:25.247432 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f1890-2e37-44fa-80a1-af9c927d5f60 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:25.252212 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7401150-0f7e-4776-8e5d-f079b7121d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:25.252948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7401150-0f7e-4776-8e5d-f079b7121d2a admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:25.253266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7401150-0f7e-4776-8e5d-f079b7121d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:25.253544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7401150-0f7e-4776-8e5d-f079b7121d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:25.254268 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7401150-0f7e-4776-8e5d-f079b7121d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:25.256634 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7401150-0f7e-4776-8e5d-f079b7121d2a admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:25.257403 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1003/2010] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:40:25.282984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1008/2011] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:40:25.283954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c46a0-7866-414e-b35d-01d0ec8bfe4a service neutron] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:25.295029 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c46a0-7866-414e-b35d-01d0ec8bfe4a service neutron] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:25.300159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2a121-6dbe-447d-976e-020746815af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:25.301060 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2a121-6dbe-447d-976e-020746815af3 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:25.301374 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2a121-6dbe-447d-976e-020746815af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:25.301648 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2a121-6dbe-447d-976e-020746815af3 admin admin] PATH_INFO: `/v3/projects/cc7dab1163434b9ca87682b719e40f4a/users/19a8d9bf84014a878ce69f0abba1e669/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:25.320048 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2a121-6dbe-447d-976e-020746815af3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc7dab1163434b9ca87682b719e40f4a, user_id=19a8d9bf84014a878ce69f0abba1e669, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:25.323380 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2a121-6dbe-447d-976e-020746815af3 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:25.345164 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1004/2012] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:25 2022] PUT /identity/v3/projects/cc7dab1163434b9ca87682b719e40f4a/users/19a8d9bf84014a878ce69f0abba1e669/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:25.357101 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d36305-a372-45e9-8a2c-8d8091f8d756 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:25.357427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d36305-a372-45e9-8a2c-8d8091f8d756 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:25.357710 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d36305-a372-45e9-8a2c-8d8091f8d756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:25.392678 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-c8d36305-a372-45e9-8a2c-8d8091f8d756 None None] MFA Rules not processed for user `19a8d9bf84014a878ce69f0abba1e669`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:25.394464 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c8d36305-a372-45e9-8a2c-8d8091f8d756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:25.490710 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1009/2013] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:25.503159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2a121-6dbe-447d-976e-020746815af3 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:25.505369 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-aea2a121-6dbe-447d-976e-020746815af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:25.581957 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2a121-6dbe-447d-976e-020746815af3 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:25.586390 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c59f7-f53f-4e98-8f87-db73a8436309 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:25.587378 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c59f7-f53f-4e98-8f87-db73a8436309 service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:25.587378 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c59f7-f53f-4e98-8f87-db73a8436309 service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:25.587643 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c59f7-f53f-4e98-8f87-db73a8436309 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:25.598007 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d36305-a372-45e9-8a2c-8d8091f8d756 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:25.599110 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c59f7-f53f-4e98-8f87-db73a8436309 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:25.600376 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c8d36305-a372-45e9-8a2c-8d8091f8d756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:25.600579 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c59f7-f53f-4e98-8f87-db73a8436309 service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:25.612175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1005/2014] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:25.633699 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c59f7-f53f-4e98-8f87-db73a8436309 service cinder] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:25.643256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:25.688510 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d36305-a372-45e9-8a2c-8d8091f8d756 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:25.694042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad96bcb-b519-4b57-8fd2-00384c0c1441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:25.694927 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad96bcb-b519-4b57-8fd2-00384c0c1441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:25.695211 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad96bcb-b519-4b57-8fd2-00384c0c1441 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:25.695522 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad96bcb-b519-4b57-8fd2-00384c0c1441 admin admin] PATH_INFO: `/v3/projects/f8d8412e0594496eb913029f47537df4` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:25.696732 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad96bcb-b519-4b57-8fd2-00384c0c1441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8d8412e0594496eb913029f47537df4)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:25.699295 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad96bcb-b519-4b57-8fd2-00384c0c1441 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:25.711272 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c59f7-f53f-4e98-8f87-db73a8436309 service cinder] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:25.727442 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:25.728394 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723e088-a3f8-47b6-a164-e14294dd9e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:25.729515 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723e088-a3f8-47b6-a164-e14294dd9e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:25.729796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723e088-a3f8-47b6-a164-e14294dd9e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:25.730622 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723e088-a3f8-47b6-a164-e14294dd9e6c admin admin] PATH_INFO: `/v3/users/19a8d9bf84014a878ce69f0abba1e669` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:25.732247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723e088-a3f8-47b6-a164-e14294dd9e6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=19a8d9bf84014a878ce69f0abba1e669)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:25.734952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723e088-a3f8-47b6-a164-e14294dd9e6c admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:25.780084 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4ad96bcb-b519-4b57-8fd2-00384c0c1441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8d8412e0594496eb913029f47537df4', 'request_id': 'req-4ad96bcb-b519-4b57-8fd2-00384c0c1441'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:25.799321 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4ad96bcb-b519-4b57-8fd2-00384c0c1441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8d8412e0594496eb913029f47537df4', 'request_id': 'req-4ad96bcb-b519-4b57-8fd2-00384c0c1441'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:25.811257 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4ad96bcb-b519-4b57-8fd2-00384c0c1441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8d8412e0594496eb913029f47537df4', 'request_id': 'req-4ad96bcb-b519-4b57-8fd2-00384c0c1441'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:25.822530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-5723e088-a3f8-47b6-a164-e14294dd9e6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19a8d9bf84014a878ce69f0abba1e669'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:25.830888 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4ad96bcb-b519-4b57-8fd2-00384c0c1441 admin admin] The token cache is being invalidate because project f8d8412e0594496eb913029f47537df4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:25.831214 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4ad96bcb-b519-4b57-8fd2-00384c0c1441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:25.831997 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4ad96bcb-b519-4b57-8fd2-00384c0c1441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:25.834769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1010/2015] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:25 2022] DELETE /identity/v3/projects/f8d8412e0594496eb913029f47537df4 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:25.841275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad96bcb-b519-4b57-8fd2-00384c0c1441 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:25.842884 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-5723e088-a3f8-47b6-a164-e14294dd9e6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19a8d9bf84014a878ce69f0abba1e669'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:25.843967 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-5723e088-a3f8-47b6-a164-e14294dd9e6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19a8d9bf84014a878ce69f0abba1e669'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:25.851077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-4ad96bcb-b519-4b57-8fd2-00384c0c1441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:25.867070 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-5723e088-a3f8-47b6-a164-e14294dd9e6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19a8d9bf84014a878ce69f0abba1e669'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:25.868985 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-5723e088-a3f8-47b6-a164-e14294dd9e6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19a8d9bf84014a878ce69f0abba1e669'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:25.907775 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1006/2016] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:25 2022] DELETE /identity/v3/users/19a8d9bf84014a878ce69f0abba1e669 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:25.908878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723e088-a3f8-47b6-a164-e14294dd9e6c admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:25.912238 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-5723e088-a3f8-47b6-a164-e14294dd9e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:26.022506 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad96bcb-b519-4b57-8fd2-00384c0c1441 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:26.110679 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723e088-a3f8-47b6-a164-e14294dd9e6c admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:26.121057 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1079232-cee9-4f1c-a3a6-6879d7893556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:26.121767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1079232-cee9-4f1c-a3a6-6879d7893556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:26.122183 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1079232-cee9-4f1c-a3a6-6879d7893556 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:26.122558 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1079232-cee9-4f1c-a3a6-6879d7893556 admin admin] PATH_INFO: `/v3/users/395023e7b4c44e2ba7e95b6fb3f3730a` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:26.124001 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1079232-cee9-4f1c-a3a6-6879d7893556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=395023e7b4c44e2ba7e95b6fb3f3730a)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:26.127966 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1079232-cee9-4f1c-a3a6-6879d7893556 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:26.172630 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3074b38-86e6-4de7-98da-b537b6d9297d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:26.174248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3074b38-86e6-4de7-98da-b537b6d9297d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:26.174717 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3074b38-86e6-4de7-98da-b537b6d9297d admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:26.175327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3074b38-86e6-4de7-98da-b537b6d9297d admin admin] PATH_INFO: `/v3/users/743eb9bdafb04335b25ec7f57dfee2de` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:26.178271 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3074b38-86e6-4de7-98da-b537b6d9297d admin admin] RBAC: Authorizing `identity:delete_user(user_id=743eb9bdafb04335b25ec7f57dfee2de)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:26.184562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3074b38-86e6-4de7-98da-b537b6d9297d admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:26.194462 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:26.243493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-f1079232-cee9-4f1c-a3a6-6879d7893556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '395023e7b4c44e2ba7e95b6fb3f3730a'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:26.263973 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-f1079232-cee9-4f1c-a3a6-6879d7893556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '395023e7b4c44e2ba7e95b6fb3f3730a'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:26.265367 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-f1079232-cee9-4f1c-a3a6-6879d7893556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '395023e7b4c44e2ba7e95b6fb3f3730a'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:26.277213 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-f1079232-cee9-4f1c-a3a6-6879d7893556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '395023e7b4c44e2ba7e95b6fb3f3730a'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:26.277981 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-f1079232-cee9-4f1c-a3a6-6879d7893556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '395023e7b4c44e2ba7e95b6fb3f3730a'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:26.293270 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-b3074b38-86e6-4de7-98da-b537b6d9297d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '743eb9bdafb04335b25ec7f57dfee2de'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:26.314054 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-b3074b38-86e6-4de7-98da-b537b6d9297d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '743eb9bdafb04335b25ec7f57dfee2de'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:26.317410 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1007/2017] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:25 2022] DELETE /identity/v3/users/395023e7b4c44e2ba7e95b6fb3f3730a => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:26.319332 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dcb6e-8f54-45dd-a730-0e4b9ff0b52d None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:26.319700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dcb6e-8f54-45dd-a730-0e4b9ff0b52d None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:26.320776 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dcb6e-8f54-45dd-a730-0e4b9ff0b52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:26.325495 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-b3074b38-86e6-4de7-98da-b537b6d9297d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '743eb9bdafb04335b25ec7f57dfee2de'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:26.331446 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:26.348569 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-510dcb6e-8f54-45dd-a730-0e4b9ff0b52d None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:26.350048 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-510dcb6e-8f54-45dd-a730-0e4b9ff0b52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:26.351765 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-b3074b38-86e6-4de7-98da-b537b6d9297d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '743eb9bdafb04335b25ec7f57dfee2de'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:26.382934 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-b3074b38-86e6-4de7-98da-b537b6d9297d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '743eb9bdafb04335b25ec7f57dfee2de'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:26.420521 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1011/2018] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:25 2022] DELETE /identity/v3/users/743eb9bdafb04335b25ec7f57dfee2de => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:26.424333 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91522752-9b66-4ab8-b6b9-f0485814c3ba None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:26.424636 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91522752-9b66-4ab8-b6b9-f0485814c3ba None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:26.425040 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91522752-9b66-4ab8-b6b9-f0485814c3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:26.474372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-91522752-9b66-4ab8-b6b9-f0485814c3ba None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:26.475766 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-91522752-9b66-4ab8-b6b9-f0485814c3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:26.534978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1008/2019] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:26.621548 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1012/2020] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:26.633766 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dcb6e-8f54-45dd-a730-0e4b9ff0b52d None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:26.636960 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-510dcb6e-8f54-45dd-a730-0e4b9ff0b52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:26.676155 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91522752-9b66-4ab8-b6b9-f0485814c3ba None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:26.678728 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-91522752-9b66-4ab8-b6b9-f0485814c3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:26.769314 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dcb6e-8f54-45dd-a730-0e4b9ff0b52d None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:26.896107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b0dbd-7310-4d24-bd2a-e410360c86ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:26.896800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b0dbd-7310-4d24-bd2a-e410360c86ac service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:26.897053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b0dbd-7310-4d24-bd2a-e410360c86ac service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:26.897388 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b0dbd-7310-4d24-bd2a-e410360c86ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:26.907498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91522752-9b66-4ab8-b6b9-f0485814c3ba None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:26.918243 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b0dbd-7310-4d24-bd2a-e410360c86ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:26.921051 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b4b56-15ec-4277-99eb-2b765cf98853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:26.923194 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b0dbd-7310-4d24-bd2a-e410360c86ac service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:26.925150 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b4b56-15ec-4277-99eb-2b765cf98853 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:26.926947 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b4b56-15ec-4277-99eb-2b765cf98853 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:26.927478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b4b56-15ec-4277-99eb-2b765cf98853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:26.947042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1009/2021] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:26.956393 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b4b56-15ec-4277-99eb-2b765cf98853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:26.958938 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b4b56-15ec-4277-99eb-2b765cf98853 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:26.979532 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1013/2022] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:40:27.591884 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b0dbd-7310-4d24-bd2a-e410360c86ac service neutron] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:27.593481 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-308b0dbd-7310-4d24-bd2a-e410360c86ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:27.614600 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:27.623151 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b0dbd-7310-4d24-bd2a-e410360c86ac service neutron] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:27.628400 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b4b56-15ec-4277-99eb-2b765cf98853 service neutron] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:27.630608 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-fa5b4b56-15ec-4277-99eb-2b765cf98853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:27.641556 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d3d6a-cae0-4f94-bab9-12a2a33ebbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:27.642426 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d3d6a-cae0-4f94-bab9-12a2a33ebbc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:27.642878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d3d6a-cae0-4f94-bab9-12a2a33ebbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:27.643307 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d3d6a-cae0-4f94-bab9-12a2a33ebbc5 admin admin] PATH_INFO: `/v3/projects/cc7dab1163434b9ca87682b719e40f4a` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:27.645691 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d3d6a-cae0-4f94-bab9-12a2a33ebbc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc7dab1163434b9ca87682b719e40f4a)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:27.651018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d3d6a-cae0-4f94-bab9-12a2a33ebbc5 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:27.663181 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b4b56-15ec-4277-99eb-2b765cf98853 service neutron] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:27.688105 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d6d7f-360e-4e7e-8a6a-c2bc96928a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:27.689170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d6d7f-360e-4e7e-8a6a-c2bc96928a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:27.689483 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d6d7f-360e-4e7e-8a6a-c2bc96928a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:27.689764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d6d7f-360e-4e7e-8a6a-c2bc96928a0f admin admin] PATH_INFO: `/v3/projects/ccffe48e478048319cc5d7ae620f8682` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:27.692002 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d6d7f-360e-4e7e-8a6a-c2bc96928a0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccffe48e478048319cc5d7ae620f8682)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:27.694059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d6d7f-360e-4e7e-8a6a-c2bc96928a0f admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:27.761899 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-7b5d3d6a-cae0-4f94-bab9-12a2a33ebbc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc7dab1163434b9ca87682b719e40f4a', 'request_id': 'req-7b5d3d6a-cae0-4f94-bab9-12a2a33ebbc5'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:27.762162 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f91d6d7f-360e-4e7e-8a6a-c2bc96928a0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccffe48e478048319cc5d7ae620f8682', 'request_id': 'req-f91d6d7f-360e-4e7e-8a6a-c2bc96928a0f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:27.770830 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f91d6d7f-360e-4e7e-8a6a-c2bc96928a0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccffe48e478048319cc5d7ae620f8682', 'request_id': 'req-f91d6d7f-360e-4e7e-8a6a-c2bc96928a0f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:27.774437 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-7b5d3d6a-cae0-4f94-bab9-12a2a33ebbc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc7dab1163434b9ca87682b719e40f4a', 'request_id': 'req-7b5d3d6a-cae0-4f94-bab9-12a2a33ebbc5'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:27.777641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f91d6d7f-360e-4e7e-8a6a-c2bc96928a0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccffe48e478048319cc5d7ae620f8682', 'request_id': 'req-f91d6d7f-360e-4e7e-8a6a-c2bc96928a0f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:27.783092 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-7b5d3d6a-cae0-4f94-bab9-12a2a33ebbc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc7dab1163434b9ca87682b719e40f4a', 'request_id': 'req-7b5d3d6a-cae0-4f94-bab9-12a2a33ebbc5'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:27.792869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f91d6d7f-360e-4e7e-8a6a-c2bc96928a0f admin admin] The token cache is being invalidate because project ccffe48e478048319cc5d7ae620f8682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:27.792869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f91d6d7f-360e-4e7e-8a6a-c2bc96928a0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:27.794259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f91d6d7f-360e-4e7e-8a6a-c2bc96928a0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:27.796401 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1014/2023] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:27 2022] DELETE /identity/v3/projects/ccffe48e478048319cc5d7ae620f8682 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:27.802533 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d6d7f-360e-4e7e-8a6a-c2bc96928a0f admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:27.804261 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-f91d6d7f-360e-4e7e-8a6a-c2bc96928a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:27.804817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-7b5d3d6a-cae0-4f94-bab9-12a2a33ebbc5 admin admin] The token cache is being invalidate because project cc7dab1163434b9ca87682b719e40f4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:27.805146 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-7b5d3d6a-cae0-4f94-bab9-12a2a33ebbc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:27.805843 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-7b5d3d6a-cae0-4f94-bab9-12a2a33ebbc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:27.812671 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1010/2024] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:27 2022] DELETE /identity/v3/projects/cc7dab1163434b9ca87682b719e40f4a => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:27.812871 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:27.816304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee6222-8db5-49e7-ad8a-d1caee4b5538 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:27.816613 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee6222-8db5-49e7-ad8a-d1caee4b5538 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:27.817136 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee6222-8db5-49e7-ad8a-d1caee4b5538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:27.841947 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-95ee6222-8db5-49e7-ad8a-d1caee4b5538 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:27.843301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-95ee6222-8db5-49e7-ad8a-d1caee4b5538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:27.895666 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d6d7f-360e-4e7e-8a6a-c2bc96928a0f admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:27.903101 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affdf6c-2b18-4631-ae91-53832d3cd079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:27.905652 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affdf6c-2b18-4631-ae91-53832d3cd079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:27.905652 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affdf6c-2b18-4631-ae91-53832d3cd079 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:27.905652 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affdf6c-2b18-4631-ae91-53832d3cd079 admin admin] PATH_INFO: `/v3/users/5bbc4e126966492fb9dc08ba8db7034b` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:27.907581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affdf6c-2b18-4631-ae91-53832d3cd079 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bbc4e126966492fb9dc08ba8db7034b)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:27.910730 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affdf6c-2b18-4631-ae91-53832d3cd079 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:27.931602 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1011/2025] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:27.940499 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d9fb8-7bfe-4b52-93ca-3f3560753082 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:27.940911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d9fb8-7bfe-4b52-93ca-3f3560753082 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:27.941284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d9fb8-7bfe-4b52-93ca-3f3560753082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:27.965609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-903d9fb8-7bfe-4b52-93ca-3f3560753082 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:27.967400 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-903d9fb8-7bfe-4b52-93ca-3f3560753082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:27.978767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1012/2026] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:27.984469 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d9fb8-7bfe-4b52-93ca-3f3560753082 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:27.999120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d9fb8-7bfe-4b52-93ca-3f3560753082 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:28.006273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4872ce-f197-4ae3-b7a9-36fe50cc8a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:28.006957 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4872ce-f197-4ae3-b7a9-36fe50cc8a06 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:28.007244 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4872ce-f197-4ae3-b7a9-36fe50cc8a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:28.007562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4872ce-f197-4ae3-b7a9-36fe50cc8a06 admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:28.008499 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4872ce-f197-4ae3-b7a9-36fe50cc8a06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:28.008813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4872ce-f197-4ae3-b7a9-36fe50cc8a06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:28.010913 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4872ce-f197-4ae3-b7a9-36fe50cc8a06 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:28.030196 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-7affdf6c-2b18-4631-ae91-53832d3cd079 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bbc4e126966492fb9dc08ba8db7034b'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:28.033627 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1013/2027] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:28.034765 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4872ce-f197-4ae3-b7a9-36fe50cc8a06 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:28.039487 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-dd4872ce-f197-4ae3-b7a9-36fe50cc8a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:28.055256 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-7affdf6c-2b18-4631-ae91-53832d3cd079 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bbc4e126966492fb9dc08ba8db7034b'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:28.056346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-7affdf6c-2b18-4631-ae91-53832d3cd079 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bbc4e126966492fb9dc08ba8db7034b'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:28.076117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4872ce-f197-4ae3-b7a9-36fe50cc8a06 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:28.099350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-7affdf6c-2b18-4631-ae91-53832d3cd079 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bbc4e126966492fb9dc08ba8db7034b'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:28.118605 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-7affdf6c-2b18-4631-ae91-53832d3cd079 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bbc4e126966492fb9dc08ba8db7034b'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:28.135396 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1015/2028] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:27 2022] DELETE /identity/v3/users/5bbc4e126966492fb9dc08ba8db7034b => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:28.135396 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affdf6c-2b18-4631-ae91-53832d3cd079 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:28.135682 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79915d4a-b960-4674-9f16-38f064cc1e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:28.139684 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79915d4a-b960-4674-9f16-38f064cc1e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:28.139969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79915d4a-b960-4674-9f16-38f064cc1e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:28.140247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79915d4a-b960-4674-9f16-38f064cc1e37 admin admin] PATH_INFO: `/v3/projects/2f674ce8b3ec41ca821506f0c0e0b714` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:28.159132 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79915d4a-b960-4674-9f16-38f064cc1e37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f674ce8b3ec41ca821506f0c0e0b714)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:28.161573 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79915d4a-b960-4674-9f16-38f064cc1e37 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:28.163646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-7affdf6c-2b18-4631-ae91-53832d3cd079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:28.267032 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-79915d4a-b960-4674-9f16-38f064cc1e37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f674ce8b3ec41ca821506f0c0e0b714', 'request_id': 'req-79915d4a-b960-4674-9f16-38f064cc1e37'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:28.290023 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-79915d4a-b960-4674-9f16-38f064cc1e37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f674ce8b3ec41ca821506f0c0e0b714', 'request_id': 'req-79915d4a-b960-4674-9f16-38f064cc1e37'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:28.306086 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:28.311291 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-79915d4a-b960-4674-9f16-38f064cc1e37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f674ce8b3ec41ca821506f0c0e0b714', 'request_id': 'req-79915d4a-b960-4674-9f16-38f064cc1e37'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:28.342298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-79915d4a-b960-4674-9f16-38f064cc1e37 admin admin] The token cache is being invalidate because project 2f674ce8b3ec41ca821506f0c0e0b714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:28.344994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-79915d4a-b960-4674-9f16-38f064cc1e37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:28.346079 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-79915d4a-b960-4674-9f16-38f064cc1e37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:28.348883 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1014/2029] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:28 2022] DELETE /identity/v3/projects/2f674ce8b3ec41ca821506f0c0e0b714 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:28.354066 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f8ee6-5a60-482b-9a2a-ebc0b0431adc None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:28.355540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f8ee6-5a60-482b-9a2a-ebc0b0431adc None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:28.356107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f8ee6-5a60-482b-9a2a-ebc0b0431adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:28.374306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affdf6c-2b18-4631-ae91-53832d3cd079 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:28.407134 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-e94f8ee6-5a60-482b-9a2a-ebc0b0431adc None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:28.409850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-e94f8ee6-5a60-482b-9a2a-ebc0b0431adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:28.426349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:28.504875 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2b238-91b8-4040-bd7c-395bea641cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:28.506234 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2b238-91b8-4040-bd7c-395bea641cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:28.506857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2b238-91b8-4040-bd7c-395bea641cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:28.507354 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2b238-91b8-4040-bd7c-395bea641cf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:28.508338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2b238-91b8-4040-bd7c-395bea641cf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:28.511260 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2b238-91b8-4040-bd7c-395bea641cf9 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:28.549539 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1015/2030] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:28.558399 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9bebe-66cd-41a0-b1b4-02155e005fe5 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:28.558698 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9bebe-66cd-41a0-b1b4-02155e005fe5 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:28.558986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9bebe-66cd-41a0-b1b4-02155e005fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:28.565753 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1016/2031] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:28 2022] POST /identity/v3/projects => generated 373 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:28.570252 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2b238-91b8-4040-bd7c-395bea641cf9 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:28.573789 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-39c2b238-91b8-4040-bd7c-395bea641cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:28.589397 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-33d9bebe-66cd-41a0-b1b4-02155e005fe5 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:28.590758 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-33d9bebe-66cd-41a0-b1b4-02155e005fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:28.701344 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2b238-91b8-4040-bd7c-395bea641cf9 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:28.708286 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1016/2032] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:28.709928 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9bebe-66cd-41a0-b1b4-02155e005fe5 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:28.710714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad224c-a370-4d3e-9194-b565177931ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:28.712075 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad224c-a370-4d3e-9194-b565177931ca admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:28.712655 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad224c-a370-4d3e-9194-b565177931ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:28.713434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad224c-a370-4d3e-9194-b565177931ca admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:28.714529 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-33d9bebe-66cd-41a0-b1b4-02155e005fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:28.715160 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad224c-a370-4d3e-9194-b565177931ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:28.718159 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad224c-a370-4d3e-9194-b565177931ca admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:28.731064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9bebe-66cd-41a0-b1b4-02155e005fe5 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:28.749365 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928edbf8-437d-4980-a8e4-d740484b3fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:28.750712 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928edbf8-437d-4980-a8e4-d740484b3fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:28.751243 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928edbf8-437d-4980-a8e4-d740484b3fce admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:28.751546 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928edbf8-437d-4980-a8e4-d740484b3fce admin admin] PATH_INFO: `/v3/projects/c83fd0ceea7240eb97520d53c438def6` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:28.753421 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928edbf8-437d-4980-a8e4-d740484b3fce admin admin] RBAC: Authorizing `identity:delete_project(project_id=c83fd0ceea7240eb97520d53c438def6)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:28.759864 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928edbf8-437d-4980-a8e4-d740484b3fce admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:28.763419 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1017/2033] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:28 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:28.764454 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad224c-a370-4d3e-9194-b565177931ca admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:28.774981 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad224c-a370-4d3e-9194-b565177931ca admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:28.780453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02aede-1aae-45c9-9896-1e430bb41c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:28.781180 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02aede-1aae-45c9-9896-1e430bb41c3a admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:28.781471 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02aede-1aae-45c9-9896-1e430bb41c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:28.781794 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02aede-1aae-45c9-9896-1e430bb41c3a admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:28.782549 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02aede-1aae-45c9-9896-1e430bb41c3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:28.782878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02aede-1aae-45c9-9896-1e430bb41c3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:28.784558 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02aede-1aae-45c9-9896-1e430bb41c3a admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:28.802401 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1018/2034] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:28.806171 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02aede-1aae-45c9-9896-1e430bb41c3a admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:28.818107 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02aede-1aae-45c9-9896-1e430bb41c3a admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:28.836895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-928edbf8-437d-4980-a8e4-d740484b3fce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c83fd0ceea7240eb97520d53c438def6', 'request_id': 'req-928edbf8-437d-4980-a8e4-d740484b3fce'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:28.849124 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-928edbf8-437d-4980-a8e4-d740484b3fce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c83fd0ceea7240eb97520d53c438def6', 'request_id': 'req-928edbf8-437d-4980-a8e4-d740484b3fce'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:28.858186 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-928edbf8-437d-4980-a8e4-d740484b3fce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c83fd0ceea7240eb97520d53c438def6', 'request_id': 'req-928edbf8-437d-4980-a8e4-d740484b3fce'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:28.879885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-928edbf8-437d-4980-a8e4-d740484b3fce admin admin] The token cache is being invalidate because project c83fd0ceea7240eb97520d53c438def6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:28.881111 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-928edbf8-437d-4980-a8e4-d740484b3fce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:28.882116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-928edbf8-437d-4980-a8e4-d740484b3fce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:28.884587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1017/2035] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:28 2022] DELETE /identity/v3/projects/c83fd0ceea7240eb97520d53c438def6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:28.886385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928edbf8-437d-4980-a8e4-d740484b3fce admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:28.889139 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-928edbf8-437d-4980-a8e4-d740484b3fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:28.896372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ecffde-512b-4f23-9638-c523cf25707d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:28.896886 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ecffde-512b-4f23-9638-c523cf25707d admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:28.897204 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ecffde-512b-4f23-9638-c523cf25707d admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:28.897477 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ecffde-512b-4f23-9638-c523cf25707d admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:28.898223 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ecffde-512b-4f23-9638-c523cf25707d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:28.899926 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ecffde-512b-4f23-9638-c523cf25707d admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:28.913013 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1019/2036] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:28.914810 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e0045-76b4-413c-a4fd-e8c0896c6041 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:28.915097 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e0045-76b4-413c-a4fd-e8c0896c6041 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:28.915375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e0045-76b4-413c-a4fd-e8c0896c6041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:28.919959 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:28.943872 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-c59e0045-76b4-413c-a4fd-e8c0896c6041 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:28.946851 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c59e0045-76b4-413c-a4fd-e8c0896c6041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:28.968524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1020/2037] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:28.968524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e0045-76b4-413c-a4fd-e8c0896c6041 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:28.968524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c59e0045-76b4-413c-a4fd-e8c0896c6041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:28.983775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e0045-76b4-413c-a4fd-e8c0896c6041 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:28.989293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f5dcc-04e4-42b1-85d7-4e48cdf5123a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:28.991485 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f5dcc-04e4-42b1-85d7-4e48cdf5123a admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:28.991485 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f5dcc-04e4-42b1-85d7-4e48cdf5123a admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:28.991636 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f5dcc-04e4-42b1-85d7-4e48cdf5123a admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:28.992313 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f5dcc-04e4-42b1-85d7-4e48cdf5123a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:28.994018 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f5dcc-04e4-42b1-85d7-4e48cdf5123a admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:29.007909 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1021/2038] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:29.009659 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94316f-e868-47f8-929a-c2204ff08f9e None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:29.009925 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94316f-e868-47f8-929a-c2204ff08f9e None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:29.011148 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94316f-e868-47f8-929a-c2204ff08f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:29.021571 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928edbf8-437d-4980-a8e4-d740484b3fce admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:29.028361 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbce2c-6f8a-4e53-ab82-6d0244ed8489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:29.029284 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbce2c-6f8a-4e53-ab82-6d0244ed8489 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:29.029778 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbce2c-6f8a-4e53-ab82-6d0244ed8489 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:29.030394 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbce2c-6f8a-4e53-ab82-6d0244ed8489 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:29.031161 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbce2c-6f8a-4e53-ab82-6d0244ed8489 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:29.033719 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbce2c-6f8a-4e53-ab82-6d0244ed8489 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:29.035637 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-0a94316f-e868-47f8-929a-c2204ff08f9e None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:29.037518 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-0a94316f-e868-47f8-929a-c2204ff08f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:29.049999 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1022/2039] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:29.053571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94316f-e868-47f8-929a-c2204ff08f9e None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:29.060071 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:29.067025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94316f-e868-47f8-929a-c2204ff08f9e None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:29.076824 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c424654-2784-41c2-b63f-0b17dc7271da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:29.078174 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c424654-2784-41c2-b63f-0b17dc7271da admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:29.078748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c424654-2784-41c2-b63f-0b17dc7271da admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:29.078879 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c424654-2784-41c2-b63f-0b17dc7271da admin admin] PATH_INFO: `/v3/projects/fcfa0fc2fe6b454da24ed8742f017e53/users/faabe2b34777437ab755ad9261f7248e/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:29.089160 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1018/2040] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:28 2022] POST /identity/v3/projects => generated 373 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:29.093162 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbce2c-6f8a-4e53-ab82-6d0244ed8489 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:29.099534 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c424654-2784-41c2-b63f-0b17dc7271da admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcfa0fc2fe6b454da24ed8742f017e53, user_id=faabe2b34777437ab755ad9261f7248e, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:29.102320 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c424654-2784-41c2-b63f-0b17dc7271da admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:29.122459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1023/2041] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:29 2022] PUT /identity/v3/projects/fcfa0fc2fe6b454da24ed8742f017e53/users/faabe2b34777437ab755ad9261f7248e/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:29.123790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c424654-2784-41c2-b63f-0b17dc7271da admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:29.176696 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbce2c-6f8a-4e53-ab82-6d0244ed8489 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:29.226365 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c424654-2784-41c2-b63f-0b17dc7271da admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:29.234193 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f35c69-7094-42b4-bef6-bfa8f2469807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:29.235646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f35c69-7094-42b4-bef6-bfa8f2469807 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:29.236310 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f35c69-7094-42b4-bef6-bfa8f2469807 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:29.236702 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f35c69-7094-42b4-bef6-bfa8f2469807 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:29.237575 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f35c69-7094-42b4-bef6-bfa8f2469807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:29.241378 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f35c69-7094-42b4-bef6-bfa8f2469807 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:29.245849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b8046-a363-429c-a08b-5aa03b1d71a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:29.246533 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b8046-a363-429c-a08b-5aa03b1d71a6 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:29.246814 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b8046-a363-429c-a08b-5aa03b1d71a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:29.247242 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b8046-a363-429c-a08b-5aa03b1d71a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:29.248167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b8046-a363-429c-a08b-5aa03b1d71a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:29.249104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b8046-a363-429c-a08b-5aa03b1d71a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:29.250306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b8046-a363-429c-a08b-5aa03b1d71a6 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:29.268120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1019/2042] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:29.273193 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1024/2043] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:29 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:29.273811 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d13b46-7d8c-4a5b-aa03-6436b49fea1f None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:29.274119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d13b46-7d8c-4a5b-aa03-6436b49fea1f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:29.274397 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d13b46-7d8c-4a5b-aa03-6436b49fea1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:29.278761 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f35c69-7094-42b4-bef6-bfa8f2469807 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:29.294760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f35c69-7094-42b4-bef6-bfa8f2469807 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:29.299518 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82492d20-aa1e-46d4-a0d8-74686a4a1124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:29.300186 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82492d20-aa1e-46d4-a0d8-74686a4a1124 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:29.300466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82492d20-aa1e-46d4-a0d8-74686a4a1124 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:29.300743 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82492d20-aa1e-46d4-a0d8-74686a4a1124 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:29.301482 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82492d20-aa1e-46d4-a0d8-74686a4a1124 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:29.303501 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82492d20-aa1e-46d4-a0d8-74686a4a1124 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:29.325493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-56d13b46-7d8c-4a5b-aa03-6436b49fea1f None None] MFA Rules not processed for user `faabe2b34777437ab755ad9261f7248e`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:29.326999 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-56d13b46-7d8c-4a5b-aa03-6436b49fea1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:29.373712 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1025/2044] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:29 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:40:29.374765 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82492d20-aa1e-46d4-a0d8-74686a4a1124 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:29.383086 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:29.424137 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1020/2045] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:29.429114 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d13b46-7d8c-4a5b-aa03-6436b49fea1f None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:29.441419 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d13b46-7d8c-4a5b-aa03-6436b49fea1f None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:29.535848 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82492d20-aa1e-46d4-a0d8-74686a4a1124 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:29.542961 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bd0e4-1434-4dd1-83cc-e180e8e791f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:29.543952 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bd0e4-1434-4dd1-83cc-e180e8e791f1 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:29.545219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bd0e4-1434-4dd1-83cc-e180e8e791f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:29.545700 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bd0e4-1434-4dd1-83cc-e180e8e791f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:29.546587 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934bd0e4-1434-4dd1-83cc-e180e8e791f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:29.548428 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934bd0e4-1434-4dd1-83cc-e180e8e791f1 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:29.556228 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef451203-32b5-41a5-a98b-b20ec007117f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:29.557681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef451203-32b5-41a5-a98b-b20ec007117f admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:29.557971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef451203-32b5-41a5-a98b-b20ec007117f admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:29.558251 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef451203-32b5-41a5-a98b-b20ec007117f admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:29.559283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef451203-32b5-41a5-a98b-b20ec007117f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:29.561869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef451203-32b5-41a5-a98b-b20ec007117f admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:29.562617 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1026/2046] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:29.563841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bd0e4-1434-4dd1-83cc-e180e8e791f1 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:29.567794 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-934bd0e4-1434-4dd1-83cc-e180e8e791f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:29.598702 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1021/2047] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:29 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:29.598702 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef451203-32b5-41a5-a98b-b20ec007117f admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:29.599521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-ef451203-32b5-41a5-a98b-b20ec007117f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:29.614997 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef451203-32b5-41a5-a98b-b20ec007117f admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:29.622077 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf68ec8-186b-482c-9078-e1eb3cd5c59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:29.624108 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf68ec8-186b-482c-9078-e1eb3cd5c59a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:29.624427 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf68ec8-186b-482c-9078-e1eb3cd5c59a admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:29.624759 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf68ec8-186b-482c-9078-e1eb3cd5c59a admin admin] PATH_INFO: `/v3/users/df4a8845aee44b2f973bd8ad45a0c027` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:29.626095 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf68ec8-186b-482c-9078-e1eb3cd5c59a admin admin] RBAC: Authorizing `identity:delete_user(user_id=df4a8845aee44b2f973bd8ad45a0c027)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:29.628055 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf68ec8-186b-482c-9078-e1eb3cd5c59a admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:29.641220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:29.664983 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bd0e4-1434-4dd1-83cc-e180e8e791f1 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:29.669991 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e6582-cb07-45c9-8485-4cc25c812995 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:29.672071 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e6582-cb07-45c9-8485-4cc25c812995 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:29.672600 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e6582-cb07-45c9-8485-4cc25c812995 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:29.672953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e6582-cb07-45c9-8485-4cc25c812995 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:29.689268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3cf68ec8-186b-482c-9078-e1eb3cd5c59a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df4a8845aee44b2f973bd8ad45a0c027'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:29.698834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3cf68ec8-186b-482c-9078-e1eb3cd5c59a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df4a8845aee44b2f973bd8ad45a0c027'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:29.699816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3cf68ec8-186b-482c-9078-e1eb3cd5c59a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df4a8845aee44b2f973bd8ad45a0c027'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:29.709370 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3cf68ec8-186b-482c-9078-e1eb3cd5c59a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df4a8845aee44b2f973bd8ad45a0c027'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:29.710208 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-3cf68ec8-186b-482c-9078-e1eb3cd5c59a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df4a8845aee44b2f973bd8ad45a0c027'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:29.735155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1022/2048] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:29 2022] DELETE /identity/v3/users/df4a8845aee44b2f973bd8ad45a0c027 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:29.736162 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf68ec8-186b-482c-9078-e1eb3cd5c59a admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:29.737926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3cf68ec8-186b-482c-9078-e1eb3cd5c59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:29.755086 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e6582-cb07-45c9-8485-4cc25c812995 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:29.757110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e6582-cb07-45c9-8485-4cc25c812995 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:29.769380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1027/2049] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:29.770372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e6582-cb07-45c9-8485-4cc25c812995 service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:29.776363 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:29.798041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e6582-cb07-45c9-8485-4cc25c812995 service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:29.834847 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf68ec8-186b-482c-9078-e1eb3cd5c59a admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:29.841970 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21afd8f-f4b9-487f-97e6-f5bec829f7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:29.842826 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21afd8f-f4b9-487f-97e6-f5bec829f7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:29.843776 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21afd8f-f4b9-487f-97e6-f5bec829f7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:29.844059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21afd8f-f4b9-487f-97e6-f5bec829f7f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:29.845428 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21afd8f-f4b9-487f-97e6-f5bec829f7f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:29.848720 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21afd8f-f4b9-487f-97e6-f5bec829f7f4 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:29.862972 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1023/2050] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:29.864404 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91feac69-56fd-42e0-a4c2-2f01237c4efc None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:29.864937 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91feac69-56fd-42e0-a4c2-2f01237c4efc None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:29.865382 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91feac69-56fd-42e0-a4c2-2f01237c4efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:29.880711 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60a2e6-0728-4fde-899c-59862b180ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:29.881984 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60a2e6-0728-4fde-899c-59862b180ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:29.882279 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60a2e6-0728-4fde-899c-59862b180ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:29.882559 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60a2e6-0728-4fde-899c-59862b180ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:29.883781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60a2e6-0728-4fde-899c-59862b180ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:29.885996 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60a2e6-0728-4fde-899c-59862b180ab5 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:29.889413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-91feac69-56fd-42e0-a4c2-2f01237c4efc None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:29.890756 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-91feac69-56fd-42e0-a4c2-2f01237c4efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:29.898474 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1028/2051] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:29.899350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60a2e6-0728-4fde-899c-59862b180ab5 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:29.901578 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1024/2052] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:29.902664 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91feac69-56fd-42e0-a4c2-2f01237c4efc None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:29.905885 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-5a60a2e6-0728-4fde-899c-59862b180ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:29.915763 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91feac69-56fd-42e0-a4c2-2f01237c4efc None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:29.920457 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60a2e6-0728-4fde-899c-59862b180ab5 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:29.922567 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f6c6c-674a-4ba9-814d-8503285ade26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:29.923629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f6c6c-674a-4ba9-814d-8503285ade26 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:29.923919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f6c6c-674a-4ba9-814d-8503285ade26 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:29.924194 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f6c6c-674a-4ba9-814d-8503285ade26 admin admin] PATH_INFO: `/v3/projects/38bdb820cf8c45379b6dced69e7cbfac/users/f8fd0fab9dc3448eb7853d809a6dcf63/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:29.927127 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a8469-4cef-4530-9f82-290792e4ca9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:29.929327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a8469-4cef-4530-9f82-290792e4ca9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:29.929778 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a8469-4cef-4530-9f82-290792e4ca9b admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:29.930207 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a8469-4cef-4530-9f82-290792e4ca9b admin admin] PATH_INFO: `/v3/users/faabe2b34777437ab755ad9261f7248e` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:29.931948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a8469-4cef-4530-9f82-290792e4ca9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=faabe2b34777437ab755ad9261f7248e)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:29.934998 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a8469-4cef-4530-9f82-290792e4ca9b admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:29.942527 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f6c6c-674a-4ba9-814d-8503285ade26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38bdb820cf8c45379b6dced69e7cbfac, user_id=f8fd0fab9dc3448eb7853d809a6dcf63, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:29.945300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f6c6c-674a-4ba9-814d-8503285ade26 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:29.968432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1025/2053] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:29 2022] PUT /identity/v3/projects/38bdb820cf8c45379b6dced69e7cbfac/users/f8fd0fab9dc3448eb7853d809a6dcf63/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:29.968432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f6c6c-674a-4ba9-814d-8503285ade26 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:29.968432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-e35f6c6c-674a-4ba9-814d-8503285ade26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:30.000344 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-a82a8469-4cef-4530-9f82-290792e4ca9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faabe2b34777437ab755ad9261f7248e'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:30.020940 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-a82a8469-4cef-4530-9f82-290792e4ca9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faabe2b34777437ab755ad9261f7248e'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:30.021707 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-a82a8469-4cef-4530-9f82-290792e4ca9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faabe2b34777437ab755ad9261f7248e'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:30.043861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-a82a8469-4cef-4530-9f82-290792e4ca9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faabe2b34777437ab755ad9261f7248e'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:30.055933 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f6c6c-674a-4ba9-814d-8503285ade26 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:30.060681 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-a82a8469-4cef-4530-9f82-290792e4ca9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faabe2b34777437ab755ad9261f7248e'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:30.063260 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a0dce-faa0-4e78-9a14-2e0bbc72a6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:30.064192 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a0dce-faa0-4e78-9a14-2e0bbc72a6bd admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.064481 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a0dce-faa0-4e78-9a14-2e0bbc72a6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.064758 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a0dce-faa0-4e78-9a14-2e0bbc72a6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.065545 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a0dce-faa0-4e78-9a14-2e0bbc72a6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:30.067289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a0dce-faa0-4e78-9a14-2e0bbc72a6bd admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:30.071343 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1029/2054] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:29 2022] DELETE /identity/v3/users/faabe2b34777437ab755ad9261f7248e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:30.071343 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a8469-4cef-4530-9f82-290792e4ca9b admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:30.073674 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-a82a8469-4cef-4530-9f82-290792e4ca9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:30.083467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1026/2055] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:30.084029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:30.084852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9ddab-d8d9-4bab-8558-8747ca6784b0 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.085191 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9ddab-d8d9-4bab-8558-8747ca6784b0 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.085497 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9ddab-d8d9-4bab-8558-8747ca6784b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.110360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-0cd9ddab-d8d9-4bab-8558-8747ca6784b0 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:30.111626 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-0cd9ddab-d8d9-4bab-8558-8747ca6784b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:30.187819 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a8469-4cef-4530-9f82-290792e4ca9b admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:30.191639 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebcc9c-a43d-4cb0-b2a6-924bd07824c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:30.192312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebcc9c-a43d-4cb0-b2a6-924bd07824c3 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.192590 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebcc9c-a43d-4cb0-b2a6-924bd07824c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.192902 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebcc9c-a43d-4cb0-b2a6-924bd07824c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.194840 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-32ebcc9c-a43d-4cb0-b2a6-924bd07824c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:30.199049 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1027/2056] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:30.200278 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3011414-ad9e-442b-a4ef-1fbe290dfc43 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.200578 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3011414-ad9e-442b-a4ef-1fbe290dfc43 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.200870 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3011414-ad9e-442b-a4ef-1fbe290dfc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.210754 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ebcc9c-a43d-4cb0-b2a6-924bd07824c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:30.212954 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ebcc9c-a43d-4cb0-b2a6-924bd07824c3 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:30.224029 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-e3011414-ad9e-442b-a4ef-1fbe290dfc43 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:30.225168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1030/2057] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:30.226021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebcc9c-a43d-4cb0-b2a6-924bd07824c3 service neutron] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:30.227412 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-32ebcc9c-a43d-4cb0-b2a6-924bd07824c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:30.229715 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-e3011414-ad9e-442b-a4ef-1fbe290dfc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:30.242027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebcc9c-a43d-4cb0-b2a6-924bd07824c3 service neutron] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:30.246404 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1028/2058] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:30.247418 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3011414-ad9e-442b-a4ef-1fbe290dfc43 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:30.247954 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e002a-e265-45c0-9d7d-74c15f3911a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:30.248849 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e002a-e265-45c0-9d7d-74c15f3911a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.249157 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e002a-e265-45c0-9d7d-74c15f3911a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.249599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e002a-e265-45c0-9d7d-74c15f3911a9 admin admin] PATH_INFO: `/v3/projects/0135207082434f3f8427a1edd53c3f7f/users/cc0344c013424c4fae11efd76a4dcf91/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.254034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-e3011414-ad9e-442b-a4ef-1fbe290dfc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:30.271705 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e002a-e265-45c0-9d7d-74c15f3911a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0135207082434f3f8427a1edd53c3f7f, user_id=cc0344c013424c4fae11efd76a4dcf91, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:30.272954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3011414-ad9e-442b-a4ef-1fbe290dfc43 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:30.274957 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e002a-e265-45c0-9d7d-74c15f3911a9 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:30.277316 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b3dbd-42bb-4642-87cb-d865a7ab5afa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:30.278337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b3dbd-42bb-4642-87cb-d865a7ab5afa service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.278813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b3dbd-42bb-4642-87cb-d865a7ab5afa service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.279128 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b3dbd-42bb-4642-87cb-d865a7ab5afa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.290795 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b3dbd-42bb-4642-87cb-d865a7ab5afa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:30.292683 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b3dbd-42bb-4642-87cb-d865a7ab5afa service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:30.294384 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1031/2059] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:30 2022] PUT /identity/v3/projects/0135207082434f3f8427a1edd53c3f7f/users/cc0344c013424c4fae11efd76a4dcf91/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:30.295135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e002a-e265-45c0-9d7d-74c15f3911a9 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:30.303859 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1029/2060] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:30.305333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fd9e6-40c6-4523-b08a-94b9d7148dda None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.305616 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fd9e6-40c6-4523-b08a-94b9d7148dda None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.305887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fd9e6-40c6-4523-b08a-94b9d7148dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.346192 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-7f5fd9e6-40c6-4523-b08a-94b9d7148dda None None] MFA Rules not processed for user `cc0344c013424c4fae11efd76a4dcf91`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:30.347491 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-7f5fd9e6-40c6-4523-b08a-94b9d7148dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:30.365698 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:30.417845 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e002a-e265-45c0-9d7d-74c15f3911a9 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:30.423439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d016e32-d2f5-46b8-9b19-d87c3aea1c4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:30.424106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d016e32-d2f5-46b8-9b19-d87c3aea1c4c service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.427118 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d016e32-d2f5-46b8-9b19-d87c3aea1c4c service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.427401 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d016e32-d2f5-46b8-9b19-d87c3aea1c4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.495935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1030/2061] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:30.502579 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fd9e6-40c6-4523-b08a-94b9d7148dda None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:30.614812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d016e32-d2f5-46b8-9b19-d87c3aea1c4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:30.617411 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d016e32-d2f5-46b8-9b19-d87c3aea1c4c service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:30.632917 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1032/2062] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:30.643106 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fd9e6-40c6-4523-b08a-94b9d7148dda None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:30.648585 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1cc5d-056e-4c2c-9e52-ac5341434b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:30.649567 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1cc5d-056e-4c2c-9e52-ac5341434b95 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.650021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1cc5d-056e-4c2c-9e52-ac5341434b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.650492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1cc5d-056e-4c2c-9e52-ac5341434b95 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.651465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1cc5d-056e-4c2c-9e52-ac5341434b95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:30.654119 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1cc5d-056e-4c2c-9e52-ac5341434b95 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:30.691251 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1031/2063] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:30 2022] POST /identity/v3/projects => generated 385 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:40:30.692747 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d016e32-d2f5-46b8-9b19-d87c3aea1c4c service neutron] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:30.711358 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d016e32-d2f5-46b8-9b19-d87c3aea1c4c service neutron] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:30.751108 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:30.788109 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8848f6-8c83-45ff-978f-29d850723562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:30.788823 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8848f6-8c83-45ff-978f-29d850723562 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.789322 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8848f6-8c83-45ff-978f-29d850723562 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.789717 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8848f6-8c83-45ff-978f-29d850723562 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.790543 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8848f6-8c83-45ff-978f-29d850723562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:30.793502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8848f6-8c83-45ff-978f-29d850723562 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:30.819453 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1033/2064] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:30 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:30.824006 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1cc5d-056e-4c2c-9e52-ac5341434b95 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:30.840339 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1cc5d-056e-4c2c-9e52-ac5341434b95 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:30.856370 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eecc86-26c9-4cc4-be9c-3305cd56fa71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:30.857291 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eecc86-26c9-4cc4-be9c-3305cd56fa71 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.861409 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eecc86-26c9-4cc4-be9c-3305cd56fa71 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.861864 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eecc86-26c9-4cc4-be9c-3305cd56fa71 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.862874 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eecc86-26c9-4cc4-be9c-3305cd56fa71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:30.865912 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eecc86-26c9-4cc4-be9c-3305cd56fa71 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:30.893117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1032/2065] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:30.894918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8848f6-8c83-45ff-978f-29d850723562 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:30.906692 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8848f6-8c83-45ff-978f-29d850723562 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:30.912398 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760382a5-8757-4f79-b8a6-0cf362886798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:30.913606 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760382a5-8757-4f79-b8a6-0cf362886798 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.913606 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760382a5-8757-4f79-b8a6-0cf362886798 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.913899 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760382a5-8757-4f79-b8a6-0cf362886798 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.914696 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760382a5-8757-4f79-b8a6-0cf362886798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:30.917028 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760382a5-8757-4f79-b8a6-0cf362886798 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:30.932987 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1034/2066] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:30.937556 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eecc86-26c9-4cc4-be9c-3305cd56fa71 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:30.950464 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760382a5-8757-4f79-b8a6-0cf362886798 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:30.951606 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eecc86-26c9-4cc4-be9c-3305cd56fa71 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:30.952983 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-760382a5-8757-4f79-b8a6-0cf362886798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:30.959495 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae6457-e7b7-40ef-85bf-e1de2115dd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:30.960636 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae6457-e7b7-40ef-85bf-e1de2115dd90 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.961001 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae6457-e7b7-40ef-85bf-e1de2115dd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.961329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae6457-e7b7-40ef-85bf-e1de2115dd90 admin admin] PATH_INFO: `/v3/projects/960de2ba31c1488cacc1c6312b8b6c36/users/446cca77db594a3f9d6e8e76d33fb63f/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.975532 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760382a5-8757-4f79-b8a6-0cf362886798 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:31.001582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae6457-e7b7-40ef-85bf-e1de2115dd90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=960de2ba31c1488cacc1c6312b8b6c36, user_id=446cca77db594a3f9d6e8e76d33fb63f, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:31.002929 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2107e54-e84b-4d86-998e-3c8e76b6bb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:31.004429 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae6457-e7b7-40ef-85bf-e1de2115dd90 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:31.006372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2107e54-e84b-4d86-998e-3c8e76b6bb77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:31.007907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2107e54-e84b-4d86-998e-3c8e76b6bb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:31.008772 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2107e54-e84b-4d86-998e-3c8e76b6bb77 admin admin] PATH_INFO: `/v3/projects/41665eb634e24e21ae2ca7cc6a87f8a2` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:31.015270 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2107e54-e84b-4d86-998e-3c8e76b6bb77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41665eb634e24e21ae2ca7cc6a87f8a2)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:31.022508 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2107e54-e84b-4d86-998e-3c8e76b6bb77 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:31.033801 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1033/2067] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:30 2022] PUT /identity/v3/projects/960de2ba31c1488cacc1c6312b8b6c36/users/446cca77db594a3f9d6e8e76d33fb63f/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:31.037653 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15f30c-6822-431f-bb1e-5af47bc50d3b None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:31.039157 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15f30c-6822-431f-bb1e-5af47bc50d3b None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:31.040847 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15f30c-6822-431f-bb1e-5af47bc50d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:31.102734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-e2107e54-e84b-4d86-998e-3c8e76b6bb77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41665eb634e24e21ae2ca7cc6a87f8a2', 'request_id': 'req-e2107e54-e84b-4d86-998e-3c8e76b6bb77'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.112033 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-3e15f30c-6822-431f-bb1e-5af47bc50d3b None None] MFA Rules not processed for user `446cca77db594a3f9d6e8e76d33fb63f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:31.114224 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-e2107e54-e84b-4d86-998e-3c8e76b6bb77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41665eb634e24e21ae2ca7cc6a87f8a2', 'request_id': 'req-e2107e54-e84b-4d86-998e-3c8e76b6bb77'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.115846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3e15f30c-6822-431f-bb1e-5af47bc50d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:31.124000 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-e2107e54-e84b-4d86-998e-3c8e76b6bb77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41665eb634e24e21ae2ca7cc6a87f8a2', 'request_id': 'req-e2107e54-e84b-4d86-998e-3c8e76b6bb77'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.142187 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-e2107e54-e84b-4d86-998e-3c8e76b6bb77 admin admin] The token cache is being invalidate because project 41665eb634e24e21ae2ca7cc6a87f8a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:31.142798 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-e2107e54-e84b-4d86-998e-3c8e76b6bb77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.143561 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-e2107e54-e84b-4d86-998e-3c8e76b6bb77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.145651 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1035/2068] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:30 2022] DELETE /identity/v3/projects/41665eb634e24e21ae2ca7cc6a87f8a2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:31.149798 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2107e54-e84b-4d86-998e-3c8e76b6bb77 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:31.151437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-e2107e54-e84b-4d86-998e-3c8e76b6bb77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:31.195474 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:31.233228 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1034/2069] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:31.236335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15f30c-6822-431f-bb1e-5af47bc50d3b None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:31.238819 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3e15f30c-6822-431f-bb1e-5af47bc50d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:31.264044 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2107e54-e84b-4d86-998e-3c8e76b6bb77 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:31.269173 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66af74-268b-4bd5-aee3-c105b2acb987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:31.270196 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66af74-268b-4bd5-aee3-c105b2acb987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:31.270560 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66af74-268b-4bd5-aee3-c105b2acb987 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:31.270835 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66af74-268b-4bd5-aee3-c105b2acb987 admin admin] PATH_INFO: `/v3/users/fed989144e1449fb9b8d2b5352407b7f` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:31.272427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66af74-268b-4bd5-aee3-c105b2acb987 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fed989144e1449fb9b8d2b5352407b7f)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:31.275022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66af74-268b-4bd5-aee3-c105b2acb987 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:31.318905 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15f30c-6822-431f-bb1e-5af47bc50d3b None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:31.321992 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:31.324287 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5ae03-6e7a-419f-ab25-3286654986e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:31.325231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5ae03-6e7a-419f-ab25-3286654986e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:31.325472 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5ae03-6e7a-419f-ab25-3286654986e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:31.325740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5ae03-6e7a-419f-ab25-3286654986e5 admin admin] PATH_INFO: `/v3/projects/fcfa0fc2fe6b454da24ed8742f017e53` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:31.326943 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5ae03-6e7a-419f-ab25-3286654986e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcfa0fc2fe6b454da24ed8742f017e53)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:31.328958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5ae03-6e7a-419f-ab25-3286654986e5 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:31.334900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-2a66af74-268b-4bd5-aee3-c105b2acb987 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fed989144e1449fb9b8d2b5352407b7f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.349061 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-2a66af74-268b-4bd5-aee3-c105b2acb987 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fed989144e1449fb9b8d2b5352407b7f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.350380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-2a66af74-268b-4bd5-aee3-c105b2acb987 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fed989144e1449fb9b8d2b5352407b7f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.367838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-2a66af74-268b-4bd5-aee3-c105b2acb987 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fed989144e1449fb9b8d2b5352407b7f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.375630 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-2a66af74-268b-4bd5-aee3-c105b2acb987 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fed989144e1449fb9b8d2b5352407b7f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.384393 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1036/2070] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:31 2022] DELETE /identity/v3/users/fed989144e1449fb9b8d2b5352407b7f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:31.386728 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66af74-268b-4bd5-aee3-c105b2acb987 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:31.388424 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-2a66af74-268b-4bd5-aee3-c105b2acb987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:31.396885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-a5f5ae03-6e7a-419f-ab25-3286654986e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcfa0fc2fe6b454da24ed8742f017e53', 'request_id': 'req-a5f5ae03-6e7a-419f-ab25-3286654986e5'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.409688 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-a5f5ae03-6e7a-419f-ab25-3286654986e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcfa0fc2fe6b454da24ed8742f017e53', 'request_id': 'req-a5f5ae03-6e7a-419f-ab25-3286654986e5'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.425620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-a5f5ae03-6e7a-419f-ab25-3286654986e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcfa0fc2fe6b454da24ed8742f017e53', 'request_id': 'req-a5f5ae03-6e7a-419f-ab25-3286654986e5'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.436500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-a5f5ae03-6e7a-419f-ab25-3286654986e5 admin admin] The token cache is being invalidate because project fcfa0fc2fe6b454da24ed8742f017e53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:31.436780 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-a5f5ae03-6e7a-419f-ab25-3286654986e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.437550 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-a5f5ae03-6e7a-419f-ab25-3286654986e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.441255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1035/2071] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:31 2022] DELETE /identity/v3/projects/fcfa0fc2fe6b454da24ed8742f017e53 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:31.443903 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade95850-b7cb-4022-b22c-f3e5b562149e None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:31.444187 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade95850-b7cb-4022-b22c-f3e5b562149e None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:31.445439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade95850-b7cb-4022-b22c-f3e5b562149e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:31.496352 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66af74-268b-4bd5-aee3-c105b2acb987 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:31.498773 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-ade95850-b7cb-4022-b22c-f3e5b562149e None None] MFA Rules not processed for user `f8fd0fab9dc3448eb7853d809a6dcf63`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:31.499668 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-ade95850-b7cb-4022-b22c-f3e5b562149e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:31.518147 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c9c53-7330-410d-8361-6496448cca6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:31.518844 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c9c53-7330-410d-8361-6496448cca6e admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:31.519128 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c9c53-7330-410d-8361-6496448cca6e admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:31.519414 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c9c53-7330-410d-8361-6496448cca6e admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:31.520130 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c9c53-7330-410d-8361-6496448cca6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:31.522119 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c9c53-7330-410d-8361-6496448cca6e admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:31.569558 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1037/2072] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:31 2022] POST /identity/v3/projects => generated 383 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:31.569558 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e5ce0-f60c-46d2-b488-95737ec4e49e None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:31.571420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e5ce0-f60c-46d2-b488-95737ec4e49e None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:31.572006 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e5ce0-f60c-46d2-b488-95737ec4e49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:31.597448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-463e5ce0-f60c-46d2-b488-95737ec4e49e None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:31.599022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-463e5ce0-f60c-46d2-b488-95737ec4e49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:31.620093 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1036/2073] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:31.620675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade95850-b7cb-4022-b22c-f3e5b562149e None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:31.622692 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-ade95850-b7cb-4022-b22c-f3e5b562149e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:31.638890 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:31.644882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade95850-b7cb-4022-b22c-f3e5b562149e None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:31.685147 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1038/2074] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:31.685147 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e5ce0-f60c-46d2-b488-95737ec4e49e None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:31.686340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-463e5ce0-f60c-46d2-b488-95737ec4e49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:31.700167 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e5ce0-f60c-46d2-b488-95737ec4e49e None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:31.709566 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dade3de-6c4c-4d2b-955d-6c4a0a7ed943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:31.710238 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dade3de-6c4c-4d2b-955d-6c4a0a7ed943 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:31.710518 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dade3de-6c4c-4d2b-955d-6c4a0a7ed943 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:31.710793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dade3de-6c4c-4d2b-955d-6c4a0a7ed943 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:31.711493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dade3de-6c4c-4d2b-955d-6c4a0a7ed943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:31.714952 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dade3de-6c4c-4d2b-955d-6c4a0a7ed943 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:31.771620 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1039/2075] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:31 2022] POST /identity/v3/projects => generated 373 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:31.773221 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe97328-270c-4ac7-a6bf-b226b89da881 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:31.773907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe97328-270c-4ac7-a6bf-b226b89da881 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:31.774798 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe97328-270c-4ac7-a6bf-b226b89da881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:31.781960 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebedfc54-a8f0-4ded-b88d-601719b4644a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:31.782877 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedfc54-a8f0-4ded-b88d-601719b4644a admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:31.783157 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedfc54-a8f0-4ded-b88d-601719b4644a admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:31.783428 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedfc54-a8f0-4ded-b88d-601719b4644a admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:31.784193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebedfc54-a8f0-4ded-b88d-601719b4644a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:31.786816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebedfc54-a8f0-4ded-b88d-601719b4644a admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:31.798039 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-1fe97328-270c-4ac7-a6bf-b226b89da881 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:31.799251 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-1fe97328-270c-4ac7-a6bf-b226b89da881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:31.812132 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1037/2076] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:31 2022] POST /identity/v3/users => generated 298 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:31.812795 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebedfc54-a8f0-4ded-b88d-601719b4644a admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:31.814704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:31.827722 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebedfc54-a8f0-4ded-b88d-601719b4644a admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:31.876416 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1040/2077] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:31.877390 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe97328-270c-4ac7-a6bf-b226b89da881 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:31.878734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-1fe97328-270c-4ac7-a6bf-b226b89da881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:31.891412 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe97328-270c-4ac7-a6bf-b226b89da881 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:31.896599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24048a18-08a9-46b1-be56-4f7da6d3ca8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:31.897520 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24048a18-08a9-46b1-be56-4f7da6d3ca8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:31.897798 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24048a18-08a9-46b1-be56-4f7da6d3ca8c admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:31.898071 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24048a18-08a9-46b1-be56-4f7da6d3ca8c admin admin] PATH_INFO: `/v3/projects/db1409067fb8471894cc2cd25e498a98` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:31.899301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24048a18-08a9-46b1-be56-4f7da6d3ca8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=db1409067fb8471894cc2cd25e498a98)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:31.901767 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24048a18-08a9-46b1-be56-4f7da6d3ca8c admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:31.902538 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8fe24-3f98-42ba-81c8-45ca91a7036a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:31.903285 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8fe24-3f98-42ba-81c8-45ca91a7036a admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:31.903756 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8fe24-3f98-42ba-81c8-45ca91a7036a admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:31.904034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8fe24-3f98-42ba-81c8-45ca91a7036a admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:31.905156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8fe24-3f98-42ba-81c8-45ca91a7036a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:31.907041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8fe24-3f98-42ba-81c8-45ca91a7036a admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:31.933215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1038/2078] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:31 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:31.933215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8fe24-3f98-42ba-81c8-45ca91a7036a admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:31.943872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8fe24-3f98-42ba-81c8-45ca91a7036a admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:31.949167 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce58ae-94e6-422a-9c3c-9217a5780596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:31.949833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce58ae-94e6-422a-9c3c-9217a5780596 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:31.950209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce58ae-94e6-422a-9c3c-9217a5780596 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:31.950386 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce58ae-94e6-422a-9c3c-9217a5780596 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:31.951127 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce58ae-94e6-422a-9c3c-9217a5780596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:31.953860 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce58ae-94e6-422a-9c3c-9217a5780596 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:31.962431 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-24048a18-08a9-46b1-be56-4f7da6d3ca8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db1409067fb8471894cc2cd25e498a98', 'request_id': 'req-24048a18-08a9-46b1-be56-4f7da6d3ca8c'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.965180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1039/2079] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:31.966064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce58ae-94e6-422a-9c3c-9217a5780596 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:31.973223 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-24048a18-08a9-46b1-be56-4f7da6d3ca8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db1409067fb8471894cc2cd25e498a98', 'request_id': 'req-24048a18-08a9-46b1-be56-4f7da6d3ca8c'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.977505 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce58ae-94e6-422a-9c3c-9217a5780596 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:31.980488 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-24048a18-08a9-46b1-be56-4f7da6d3ca8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db1409067fb8471894cc2cd25e498a98', 'request_id': 'req-24048a18-08a9-46b1-be56-4f7da6d3ca8c'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.992039 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-24048a18-08a9-46b1-be56-4f7da6d3ca8c admin admin] The token cache is being invalidate because project db1409067fb8471894cc2cd25e498a98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:31.992326 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-24048a18-08a9-46b1-be56-4f7da6d3ca8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.992989 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-24048a18-08a9-46b1-be56-4f7da6d3ca8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:31.994722 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1041/2080] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:31 2022] DELETE /identity/v3/projects/db1409067fb8471894cc2cd25e498a98 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:31.999697 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24048a18-08a9-46b1-be56-4f7da6d3ca8c admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:32.001235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-24048a18-08a9-46b1-be56-4f7da6d3ca8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:32.037845 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b4dfd-f6c5-45a4-857c-937c0b99d8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:32.038514 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b4dfd-f6c5-45a4-857c-937c0b99d8e3 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.038802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b4dfd-f6c5-45a4-857c-937c0b99d8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.039081 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b4dfd-f6c5-45a4-857c-937c0b99d8e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.039820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b4dfd-f6c5-45a4-857c-937c0b99d8e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:32.040134 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b4dfd-f6c5-45a4-857c-937c0b99d8e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:32.042444 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b4dfd-f6c5-45a4-857c-937c0b99d8e3 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:32.055052 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1040/2081] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:32.056033 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b4dfd-f6c5-45a4-857c-937c0b99d8e3 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:32.057901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-030b4dfd-f6c5-45a4-857c-937c0b99d8e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:32.070695 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b4dfd-f6c5-45a4-857c-937c0b99d8e3 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:32.071313 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24048a18-08a9-46b1-be56-4f7da6d3ca8c admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:32.076111 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c8655-b715-4642-a9b3-29008a5325fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:32.076931 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c8655-b715-4642-a9b3-29008a5325fc admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.077306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c8655-b715-4642-a9b3-29008a5325fc admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.077584 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c8655-b715-4642-a9b3-29008a5325fc admin admin] PATH_INFO: `/v3/projects/62b499ef959d4109913f52de01f30a59/users/09897d75f24843d185a7cdde0e3fedc6/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.077777 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4aca4-44c3-4a44-ad08-949a3dae1bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:32.078470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4aca4-44c3-4a44-ad08-949a3dae1bbc admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.078747 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4aca4-44c3-4a44-ad08-949a3dae1bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.079015 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4aca4-44c3-4a44-ad08-949a3dae1bbc admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.079838 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4aca4-44c3-4a44-ad08-949a3dae1bbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:32.079952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:32.081826 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4aca4-44c3-4a44-ad08-949a3dae1bbc admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:32.093844 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1042/2082] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:32.095383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d357f-01f3-4de9-9aa9-f7b7c8e39d18 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.095674 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d357f-01f3-4de9-9aa9-f7b7c8e39d18 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.095960 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d357f-01f3-4de9-9aa9-f7b7c8e39d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.096096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c8655-b715-4642-a9b3-29008a5325fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=62b499ef959d4109913f52de01f30a59, user_id=09897d75f24843d185a7cdde0e3fedc6, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:32.098990 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c8655-b715-4642-a9b3-29008a5325fc admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:32.112990 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1041/2083] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:32 2022] PUT /identity/v3/projects/62b499ef959d4109913f52de01f30a59/users/09897d75f24843d185a7cdde0e3fedc6/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:32.113858 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c8655-b715-4642-a9b3-29008a5325fc admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:32.114704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-d97d357f-01f3-4de9-9aa9-f7b7c8e39d18 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:32.115402 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-632c8655-b715-4642-a9b3-29008a5325fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:32.115866 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d97d357f-01f3-4de9-9aa9-f7b7c8e39d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:32.150090 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:32.178800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1043/2084] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:32.179627 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d357f-01f3-4de9-9aa9-f7b7c8e39d18 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:32.183294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c8655-b715-4642-a9b3-29008a5325fc admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:32.188457 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5264ab-3ee3-4c3c-9652-30bdaaa53534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:32.189148 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5264ab-3ee3-4c3c-9652-30bdaaa53534 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.189435 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5264ab-3ee3-4c3c-9652-30bdaaa53534 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.189768 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5264ab-3ee3-4c3c-9652-30bdaaa53534 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.190326 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d357f-01f3-4de9-9aa9-f7b7c8e39d18 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:32.190501 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5264ab-3ee3-4c3c-9652-30bdaaa53534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:32.192434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5264ab-3ee3-4c3c-9652-30bdaaa53534 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:32.195170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3945eba-a7b5-4795-a66f-158f6eac3e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:32.196086 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3945eba-a7b5-4795-a66f-158f6eac3e58 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.196360 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3945eba-a7b5-4795-a66f-158f6eac3e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.197005 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3945eba-a7b5-4795-a66f-158f6eac3e58 admin admin] PATH_INFO: `/v3/projects/8b29cb1663a640dcbcfdd8cfcbb22982/users/883d74fad6934dc8a11200452c69b4cd/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.212391 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3945eba-a7b5-4795-a66f-158f6eac3e58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b29cb1663a640dcbcfdd8cfcbb22982, user_id=883d74fad6934dc8a11200452c69b4cd, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:32.215151 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3945eba-a7b5-4795-a66f-158f6eac3e58 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:32.220159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1042/2085] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:32 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:32.221247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5264ab-3ee3-4c3c-9652-30bdaaa53534 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:32.232815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1044/2086] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:32 2022] PUT /identity/v3/projects/8b29cb1663a640dcbcfdd8cfcbb22982/users/883d74fad6934dc8a11200452c69b4cd/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:32.234361 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df06643-814f-4e5c-9866-b3285a13a174 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.234655 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df06643-814f-4e5c-9866-b3285a13a174 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.234989 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df06643-814f-4e5c-9866-b3285a13a174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.254708 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-1df06643-814f-4e5c-9866-b3285a13a174 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:32.255932 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-1df06643-814f-4e5c-9866-b3285a13a174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:32.289435 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5264ab-3ee3-4c3c-9652-30bdaaa53534 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:32.317166 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1045/2087] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:32.317899 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df06643-814f-4e5c-9866-b3285a13a174 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:32.329216 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df06643-814f-4e5c-9866-b3285a13a174 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:32.333802 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97432690-710a-4927-a8ed-6e8fc2bd48e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:32.334458 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97432690-710a-4927-a8ed-6e8fc2bd48e8 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.334741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97432690-710a-4927-a8ed-6e8fc2bd48e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.335062 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97432690-710a-4927-a8ed-6e8fc2bd48e8 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.335755 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97432690-710a-4927-a8ed-6e8fc2bd48e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:32.337716 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97432690-710a-4927-a8ed-6e8fc2bd48e8 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:32.348790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253956b2-86e7-4d19-80c5-a02b151fa852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:32.349503 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253956b2-86e7-4d19-80c5-a02b151fa852 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.349794 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253956b2-86e7-4d19-80c5-a02b151fa852 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.350231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253956b2-86e7-4d19-80c5-a02b151fa852 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.350941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253956b2-86e7-4d19-80c5-a02b151fa852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:32.352615 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253956b2-86e7-4d19-80c5-a02b151fa852 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:32.359673 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1046/2088] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:32 2022] POST /identity/v3/users => generated 288 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:32.360608 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97432690-710a-4927-a8ed-6e8fc2bd48e8 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:32.366374 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1043/2089] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:32.366374 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253956b2-86e7-4d19-80c5-a02b151fa852 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:32.371899 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97432690-710a-4927-a8ed-6e8fc2bd48e8 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:32.376149 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253956b2-86e7-4d19-80c5-a02b151fa852 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:32.377290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea808de-b440-4cc4-8be2-4f1d9e5554d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:32.377967 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea808de-b440-4cc4-8be2-4f1d9e5554d8 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.378252 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea808de-b440-4cc4-8be2-4f1d9e5554d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.378533 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea808de-b440-4cc4-8be2-4f1d9e5554d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.379268 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea808de-b440-4cc4-8be2-4f1d9e5554d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:32.380874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea808de-b440-4cc4-8be2-4f1d9e5554d8 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:32.381357 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680387a-92b7-45e4-bf69-84d69c635cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:32.382003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680387a-92b7-45e4-bf69-84d69c635cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.382291 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680387a-92b7-45e4-bf69-84d69c635cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.382573 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680387a-92b7-45e4-bf69-84d69c635cd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.383303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680387a-92b7-45e4-bf69-84d69c635cd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:32.383615 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680387a-92b7-45e4-bf69-84d69c635cd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:32.385274 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680387a-92b7-45e4-bf69-84d69c635cd6 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:32.391781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1047/2090] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:32.392301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:32.392930 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea808de-b440-4cc4-8be2-4f1d9e5554d8 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:32.401799 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1044/2091] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:32.402830 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680387a-92b7-45e4-bf69-84d69c635cd6 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:32.407256 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea808de-b440-4cc4-8be2-4f1d9e5554d8 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:32.412527 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b72205-45a5-4c21-bd7c-e18a1307ba5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:32.412777 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680387a-92b7-45e4-bf69-84d69c635cd6 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:32.413312 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b72205-45a5-4c21-bd7c-e18a1307ba5b admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.413727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b72205-45a5-4c21-bd7c-e18a1307ba5b admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.413959 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b72205-45a5-4c21-bd7c-e18a1307ba5b admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.414943 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b72205-45a5-4c21-bd7c-e18a1307ba5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:32.416938 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b72205-45a5-4c21-bd7c-e18a1307ba5b admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:32.418396 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d933881-d3c3-485c-9006-d64fead4e47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:32.419285 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d933881-d3c3-485c-9006-d64fead4e47b admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.419575 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d933881-d3c3-485c-9006-d64fead4e47b admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.419830 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d933881-d3c3-485c-9006-d64fead4e47b admin admin] PATH_INFO: `/v3/projects/62b499ef959d4109913f52de01f30a59/users/09897d75f24843d185a7cdde0e3fedc6/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.421864 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d933881-d3c3-485c-9006-d64fead4e47b admin admin] RBAC: Authorizing `identity:create_grant(project_id=62b499ef959d4109913f52de01f30a59, user_id=09897d75f24843d185a7cdde0e3fedc6, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:32.424782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d933881-d3c3-485c-9006-d64fead4e47b admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:32.430666 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1048/2092] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:32.434198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b72205-45a5-4c21-bd7c-e18a1307ba5b admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:32.437539 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1045/2093] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:32 2022] PUT /identity/v3/projects/62b499ef959d4109913f52de01f30a59/users/09897d75f24843d185a7cdde0e3fedc6/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:32.438553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d933881-d3c3-485c-9006-d64fead4e47b admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:32.451717 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b72205-45a5-4c21-bd7c-e18a1307ba5b admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:32.514213 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841cd37-5112-48f5-863c-752610b28fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:32.515133 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841cd37-5112-48f5-863c-752610b28fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.515403 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841cd37-5112-48f5-863c-752610b28fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.515683 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841cd37-5112-48f5-863c-752610b28fdd admin admin] PATH_INFO: `/v3/projects/8b29cb1663a640dcbcfdd8cfcbb22982/users/883d74fad6934dc8a11200452c69b4cd/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.517746 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841cd37-5112-48f5-863c-752610b28fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b29cb1663a640dcbcfdd8cfcbb22982, user_id=883d74fad6934dc8a11200452c69b4cd, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:32.518839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d933881-d3c3-485c-9006-d64fead4e47b admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:32.520416 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841cd37-5112-48f5-863c-752610b28fdd admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:32.523877 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb0ba1-ae41-4f68-bd79-bb996664c888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:32.524596 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb0ba1-ae41-4f68-bd79-bb996664c888 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.524884 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb0ba1-ae41-4f68-bd79-bb996664c888 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.525200 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb0ba1-ae41-4f68-bd79-bb996664c888 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.525968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb0ba1-ae41-4f68-bd79-bb996664c888 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:32.527936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb0ba1-ae41-4f68-bd79-bb996664c888 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:32.531043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1049/2094] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:32 2022] PUT /identity/v3/projects/8b29cb1663a640dcbcfdd8cfcbb22982/users/883d74fad6934dc8a11200452c69b4cd/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:32.531920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841cd37-5112-48f5-863c-752610b28fdd admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:32.542513 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:32.566061 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1046/2095] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:32 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:32.566061 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be554c5b-af3d-420a-92fc-a9fe7f030c4d None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.566061 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be554c5b-af3d-420a-92fc-a9fe7f030c4d None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.566061 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be554c5b-af3d-420a-92fc-a9fe7f030c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.593956 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-be554c5b-af3d-420a-92fc-a9fe7f030c4d None None] MFA Rules not processed for user `09897d75f24843d185a7cdde0e3fedc6`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:32.595435 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-be554c5b-af3d-420a-92fc-a9fe7f030c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:32.615186 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841cd37-5112-48f5-863c-752610b28fdd admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:32.677455 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1047/2096] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:32.677455 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be554c5b-af3d-420a-92fc-a9fe7f030c4d None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:32.701435 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0af8a0-f6be-49b7-8cb0-e0587823e496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:32.702213 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0af8a0-f6be-49b7-8cb0-e0587823e496 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.702508 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0af8a0-f6be-49b7-8cb0-e0587823e496 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.702780 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0af8a0-f6be-49b7-8cb0-e0587823e496 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.703524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0af8a0-f6be-49b7-8cb0-e0587823e496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:32.705211 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0af8a0-f6be-49b7-8cb0-e0587823e496 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:32.729891 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1050/2097] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:32.730878 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0af8a0-f6be-49b7-8cb0-e0587823e496 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:32.733249 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-0b0af8a0-f6be-49b7-8cb0-e0587823e496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:32.755376 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:32.768436 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be554c5b-af3d-420a-92fc-a9fe7f030c4d None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:32.775626 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30cf17-ad2a-497a-917c-28904c976a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:32.777010 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30cf17-ad2a-497a-917c-28904c976a10 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.777557 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30cf17-ad2a-497a-917c-28904c976a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.777879 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30cf17-ad2a-497a-917c-28904c976a10 admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.778578 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30cf17-ad2a-497a-917c-28904c976a10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:32.780849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30cf17-ad2a-497a-917c-28904c976a10 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:32.815770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1048/2098] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:32 2022] POST /identity/v3/users => generated 283 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:32.816895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30cf17-ad2a-497a-917c-28904c976a10 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:32.827093 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0af8a0-f6be-49b7-8cb0-e0587823e496 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:32.828865 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30cf17-ad2a-497a-917c-28904c976a10 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:32.831415 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5221b10-39fd-40de-90ac-1880d2484195 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:32.832201 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5221b10-39fd-40de-90ac-1880d2484195 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.832489 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5221b10-39fd-40de-90ac-1880d2484195 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.833281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5221b10-39fd-40de-90ac-1880d2484195 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.834840 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-a5221b10-39fd-40de-90ac-1880d2484195 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:32.836537 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd2621-92f8-4422-9ccc-ffc5bfe5ac18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:32.838570 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd2621-92f8-4422-9ccc-ffc5bfe5ac18 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.838901 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd2621-92f8-4422-9ccc-ffc5bfe5ac18 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.839153 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd2621-92f8-4422-9ccc-ffc5bfe5ac18 admin admin] PATH_INFO: `/v3/projects/c84341691bfd4342a1460e6c27a34df6/users/57cc4cf540d54483b498c47f9a55efcb/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.869121 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd2621-92f8-4422-9ccc-ffc5bfe5ac18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84341691bfd4342a1460e6c27a34df6, user_id=57cc4cf540d54483b498c47f9a55efcb, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:32.872054 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd2621-92f8-4422-9ccc-ffc5bfe5ac18 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:32.894815 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1049/2099] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:32 2022] PUT /identity/v3/projects/c84341691bfd4342a1460e6c27a34df6/users/57cc4cf540d54483b498c47f9a55efcb/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:32.895822 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd2621-92f8-4422-9ccc-ffc5bfe5ac18 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:32.946338 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5221b10-39fd-40de-90ac-1880d2484195 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:32.949182 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5221b10-39fd-40de-90ac-1880d2484195 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:32.977173 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd2621-92f8-4422-9ccc-ffc5bfe5ac18 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:32.977828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1051/2100] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:32.981974 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96787ea-f5bd-4632-8f44-d95a8534bbc9 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.982633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96787ea-f5bd-4632-8f44-d95a8534bbc9 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.984816 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96787ea-f5bd-4632-8f44-d95a8534bbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.000542 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:33.003539 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1468fd7-37aa-46d1-98b5-238148f5b6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:33.004364 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1468fd7-37aa-46d1-98b5-238148f5b6cf admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.004797 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1468fd7-37aa-46d1-98b5-238148f5b6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.005180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1468fd7-37aa-46d1-98b5-238148f5b6cf admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.005968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1468fd7-37aa-46d1-98b5-238148f5b6cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:33.007689 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1468fd7-37aa-46d1-98b5-238148f5b6cf admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:33.021670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1050/2101] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:33.025307 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1468fd7-37aa-46d1-98b5-238148f5b6cf admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:33.027037 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-a96787ea-f5bd-4632-8f44-d95a8534bbc9 None None] MFA Rules not processed for user `57cc4cf540d54483b498c47f9a55efcb`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:33.028413 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-a96787ea-f5bd-4632-8f44-d95a8534bbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:33.038016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1468fd7-37aa-46d1-98b5-238148f5b6cf admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:33.045727 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a91265-5e7b-46f8-a310-7b57f8114159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:33.046518 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a91265-5e7b-46f8-a310-7b57f8114159 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.046869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a91265-5e7b-46f8-a310-7b57f8114159 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.047218 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a91265-5e7b-46f8-a310-7b57f8114159 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.048331 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a91265-5e7b-46f8-a310-7b57f8114159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:33.050334 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a91265-5e7b-46f8-a310-7b57f8114159 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:33.062559 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1051/2102] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:40:33.066316 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a91265-5e7b-46f8-a310-7b57f8114159 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:33.085864 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a91265-5e7b-46f8-a310-7b57f8114159 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:33.091778 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59a577-4ca5-4d1f-97b4-7da93efb1aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:33.093002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59a577-4ca5-4d1f-97b4-7da93efb1aa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.093477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59a577-4ca5-4d1f-97b4-7da93efb1aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.094087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59a577-4ca5-4d1f-97b4-7da93efb1aa4 admin admin] PATH_INFO: `/v3/projects/bdf3423289504a9191c6ea15be9bb6de/users/3e5bffa3e665482584a466ea1a0dcfa9/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.121580 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59a577-4ca5-4d1f-97b4-7da93efb1aa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdf3423289504a9191c6ea15be9bb6de, user_id=3e5bffa3e665482584a466ea1a0dcfa9, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:33.126197 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59a577-4ca5-4d1f-97b4-7da93efb1aa4 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:33.140862 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1052/2103] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:33.148104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1052/2104] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:33 2022] PUT /identity/v3/projects/bdf3423289504a9191c6ea15be9bb6de/users/3e5bffa3e665482584a466ea1a0dcfa9/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:33.154074 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96787ea-f5bd-4632-8f44-d95a8534bbc9 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:33.156479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-a96787ea-f5bd-4632-8f44-d95a8534bbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:33.159882 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77161356-a805-4946-bd20-9d2f718d59f3 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.160597 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77161356-a805-4946-bd20-9d2f718d59f3 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.160979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77161356-a805-4946-bd20-9d2f718d59f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.198184 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-77161356-a805-4946-bd20-9d2f718d59f3 None None] MFA Rules not processed for user `3e5bffa3e665482584a466ea1a0dcfa9`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:33.199946 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-77161356-a805-4946-bd20-9d2f718d59f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:33.266861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96787ea-f5bd-4632-8f44-d95a8534bbc9 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:33.303413 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1053/2105] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:33 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:33.308205 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77161356-a805-4946-bd20-9d2f718d59f3 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:33.325697 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:33.353812 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85acbc2-4468-4bbd-a5a0-c26a96b37606 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:33.355033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85acbc2-4468-4bbd-a5a0-c26a96b37606 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.355925 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85acbc2-4468-4bbd-a5a0-c26a96b37606 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.356532 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85acbc2-4468-4bbd-a5a0-c26a96b37606 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.432810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77161356-a805-4946-bd20-9d2f718d59f3 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:33.440554 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e7cff-390e-4526-a958-3c40f2b556b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:33.441767 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e7cff-390e-4526-a958-3c40f2b556b8 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.442244 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e7cff-390e-4526-a958-3c40f2b556b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.442723 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e7cff-390e-4526-a958-3c40f2b556b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.443702 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e7cff-390e-4526-a958-3c40f2b556b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:33.446566 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e7cff-390e-4526-a958-3c40f2b556b8 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:33.450726 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85acbc2-4468-4bbd-a5a0-c26a96b37606 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:33.452502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85acbc2-4468-4bbd-a5a0-c26a96b37606 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:33.464307 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1053/2106] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:33.466175 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666caee6-62f6-4321-9841-3f37ac861427 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.466656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666caee6-62f6-4321-9841-3f37ac861427 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.467437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666caee6-62f6-4321-9841-3f37ac861427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.480903 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:33.502186 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-666caee6-62f6-4321-9841-3f37ac861427 None None] MFA Rules not processed for user `883d74fad6934dc8a11200452c69b4cd`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:33.502425 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1054/2107] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:33 2022] POST /identity/v3/projects => generated 365 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:33.502535 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-666caee6-62f6-4321-9841-3f37ac861427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:33.504244 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e7cff-390e-4526-a958-3c40f2b556b8 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:33.599559 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e7cff-390e-4526-a958-3c40f2b556b8 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:33.605694 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c483b-fde4-44d2-ae5e-c3e36dfa58cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:33.606560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c483b-fde4-44d2-ae5e-c3e36dfa58cc admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.606850 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c483b-fde4-44d2-ae5e-c3e36dfa58cc admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.607135 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c483b-fde4-44d2-ae5e-c3e36dfa58cc admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.608247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c483b-fde4-44d2-ae5e-c3e36dfa58cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:33.611113 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c483b-fde4-44d2-ae5e-c3e36dfa58cc admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:33.614646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1054/2108] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:33.628694 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666caee6-62f6-4321-9841-3f37ac861427 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:33.641493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1055/2109] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:33 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:33.644937 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c483b-fde4-44d2-ae5e-c3e36dfa58cc admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:33.663187 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c483b-fde4-44d2-ae5e-c3e36dfa58cc admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:33.672209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44eed4a-9319-419e-8582-4371d7056aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:33.673041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44eed4a-9319-419e-8582-4371d7056aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.673496 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44eed4a-9319-419e-8582-4371d7056aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.674356 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44eed4a-9319-419e-8582-4371d7056aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.674749 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44eed4a-9319-419e-8582-4371d7056aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:33.676373 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44eed4a-9319-419e-8582-4371d7056aa1 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:33.689433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1056/2110] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:40:33.694024 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44eed4a-9319-419e-8582-4371d7056aa1 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:33.705950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44eed4a-9319-419e-8582-4371d7056aa1 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:33.706115 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666caee6-62f6-4321-9841-3f37ac861427 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:33.709742 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4de35-7092-404f-9279-853b130b40c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:33.711261 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4de35-7092-404f-9279-853b130b40c8 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.711261 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4de35-7092-404f-9279-853b130b40c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.711261 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4de35-7092-404f-9279-853b130b40c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.711607 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bff67f-391c-47be-ab5a-e25a8eea0916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:33.712758 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bff67f-391c-47be-ab5a-e25a8eea0916 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.712895 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-5ca4de35-7092-404f-9279-853b130b40c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:33.713118 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bff67f-391c-47be-ab5a-e25a8eea0916 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.713638 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bff67f-391c-47be-ab5a-e25a8eea0916 admin admin] PATH_INFO: `/v3/projects/f4b5aefc436446ac99ff3dcd154bf2b5/users/5c60b3e3a24046a8ad1343ad64c57f1b/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.732981 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bff67f-391c-47be-ab5a-e25a8eea0916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b5aefc436446ac99ff3dcd154bf2b5, user_id=5c60b3e3a24046a8ad1343ad64c57f1b, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:33.736215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bff67f-391c-47be-ab5a-e25a8eea0916 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:33.753589 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1057/2111] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:33 2022] PUT /identity/v3/projects/f4b5aefc436446ac99ff3dcd154bf2b5/users/5c60b3e3a24046a8ad1343ad64c57f1b/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:33.757623 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bff67f-391c-47be-ab5a-e25a8eea0916 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:33.786709 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca4de35-7092-404f-9279-853b130b40c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:33.788418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca4de35-7092-404f-9279-853b130b40c8 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:33.800202 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1055/2112] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:33.825913 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:33.848114 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bff67f-391c-47be-ab5a-e25a8eea0916 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:33.855710 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd94c8e-9655-43ec-ac60-95ca6770133a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:33.856499 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd94c8e-9655-43ec-ac60-95ca6770133a admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.856790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd94c8e-9655-43ec-ac60-95ca6770133a admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.857083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd94c8e-9655-43ec-ac60-95ca6770133a admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.858039 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd94c8e-9655-43ec-ac60-95ca6770133a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:33.859983 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd94c8e-9655-43ec-ac60-95ca6770133a admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:33.872627 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1058/2113] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:40:33.876704 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4de35-7092-404f-9279-853b130b40c8 service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:33.893536 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4de35-7092-404f-9279-853b130b40c8 service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:33.899816 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed17390-00de-4768-989f-4d157d96486a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:33.900955 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed17390-00de-4768-989f-4d157d96486a admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.901447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed17390-00de-4768-989f-4d157d96486a admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.901875 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed17390-00de-4768-989f-4d157d96486a admin admin] PATH_INFO: `/v3/projects/f4b5aefc436446ac99ff3dcd154bf2b5/users/5c60b3e3a24046a8ad1343ad64c57f1b/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.904489 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed17390-00de-4768-989f-4d157d96486a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b5aefc436446ac99ff3dcd154bf2b5, user_id=5c60b3e3a24046a8ad1343ad64c57f1b, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:33.907624 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed17390-00de-4768-989f-4d157d96486a admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:33.925134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1056/2114] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:33 2022] PUT /identity/v3/projects/f4b5aefc436446ac99ff3dcd154bf2b5/users/5c60b3e3a24046a8ad1343ad64c57f1b/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:33.944732 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafba6ae-d1f2-43b7-8d02-11dcb6abd6cd None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.945243 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafba6ae-d1f2-43b7-8d02-11dcb6abd6cd None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.945724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafba6ae-d1f2-43b7-8d02-11dcb6abd6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.985287 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-fafba6ae-d1f2-43b7-8d02-11dcb6abd6cd None None] MFA Rules not processed for user `5c60b3e3a24046a8ad1343ad64c57f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:33.986523 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-fafba6ae-d1f2-43b7-8d02-11dcb6abd6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:34.078323 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1059/2115] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:34.091238 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed17390-00de-4768-989f-4d157d96486a admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:34.134055 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:34.167941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed17390-00de-4768-989f-4d157d96486a admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:34.171818 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f4adc-904b-4f30-a9af-023df141c249 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:34.172448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f4adc-904b-4f30-a9af-023df141c249 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:34.172731 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f4adc-904b-4f30-a9af-023df141c249 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:34.173015 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f4adc-904b-4f30-a9af-023df141c249 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:34.182794 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f4adc-904b-4f30-a9af-023df141c249 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:34.184421 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f4adc-904b-4f30-a9af-023df141c249 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:34.196136 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1057/2116] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:34 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:40:34.273040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafba6ae-d1f2-43b7-8d02-11dcb6abd6cd None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:34.285111 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafba6ae-d1f2-43b7-8d02-11dcb6abd6cd None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:34.289399 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a7473-2d8c-4e8c-801b-d78abe61a60e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:34.290057 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a7473-2d8c-4e8c-801b-d78abe61a60e service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:34.290340 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a7473-2d8c-4e8c-801b-d78abe61a60e service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:34.290613 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a7473-2d8c-4e8c-801b-d78abe61a60e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:34.353015 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a7473-2d8c-4e8c-801b-d78abe61a60e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:34.354967 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a7473-2d8c-4e8c-801b-d78abe61a60e service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:34.364648 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1060/2117] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:34 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:34.655022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f4adc-904b-4f30-a9af-023df141c249 service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:34.665713 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f4adc-904b-4f30-a9af-023df141c249 service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:34.671771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4af2e3-1d2f-4469-959b-5ffd210768cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:34.672614 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4af2e3-1d2f-4469-959b-5ffd210768cc service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:34.673031 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4af2e3-1d2f-4469-959b-5ffd210768cc service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:34.673368 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4af2e3-1d2f-4469-959b-5ffd210768cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:34.675978 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-2c4af2e3-1d2f-4469-959b-5ffd210768cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:34.746305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4af2e3-1d2f-4469-959b-5ffd210768cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:34.747830 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4af2e3-1d2f-4469-959b-5ffd210768cc service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:34.757908 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1058/2118] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:35.296255 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a7473-2d8c-4e8c-801b-d78abe61a60e service cinder] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:35.308315 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a7473-2d8c-4e8c-801b-d78abe61a60e service cinder] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:35.312241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c5175-48d0-4fe4-9f8b-dd7cdae5b0b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:35.312914 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c5175-48d0-4fe4-9f8b-dd7cdae5b0b0 service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:35.313241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c5175-48d0-4fe4-9f8b-dd7cdae5b0b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:35.313522 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c5175-48d0-4fe4-9f8b-dd7cdae5b0b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:35.375195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4af2e3-1d2f-4469-959b-5ffd210768cc service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:35.380534 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c5175-48d0-4fe4-9f8b-dd7cdae5b0b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:35.381950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c5175-48d0-4fe4-9f8b-dd7cdae5b0b0 service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:35.383525 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4af2e3-1d2f-4469-959b-5ffd210768cc service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:35.388092 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7bbc7-4199-40a3-82a6-629b3a349709 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:35.388289 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7bbc7-4199-40a3-82a6-629b3a349709 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:35.388572 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7bbc7-4199-40a3-82a6-629b3a349709 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:35.388861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7bbc7-4199-40a3-82a6-629b3a349709 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:35.392551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1061/2119] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:35 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:35.468143 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a7bbc7-4199-40a3-82a6-629b3a349709 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:35.470069 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a7bbc7-4199-40a3-82a6-629b3a349709 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:35.485101 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1059/2120] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:44.690342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c5175-48d0-4fe4-9f8b-dd7cdae5b0b0 service cinder] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:44.696233 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:44.726292 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7bbc7-4199-40a3-82a6-629b3a349709 service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:44.764455 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c5175-48d0-4fe4-9f8b-dd7cdae5b0b0 service cinder] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:44.769138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40efa0-1136-4286-bfce-acd4660ac49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:44.769949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40efa0-1136-4286-bfce-acd4660ac49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:44.770295 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40efa0-1136-4286-bfce-acd4660ac49a admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:44.770579 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40efa0-1136-4286-bfce-acd4660ac49a admin admin] PATH_INFO: `/v3/users/3e5bffa3e665482584a466ea1a0dcfa9` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:44.771902 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40efa0-1136-4286-bfce-acd4660ac49a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e5bffa3e665482584a466ea1a0dcfa9)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:44.773938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40efa0-1136-4286-bfce-acd4660ac49a admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:44.796622 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7bbc7-4199-40a3-82a6-629b3a349709 service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:44.801614 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ed6d2-ab95-40e6-912d-819cbafe36e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:44.802634 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ed6d2-ab95-40e6-912d-819cbafe36e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:44.802961 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ed6d2-ab95-40e6-912d-819cbafe36e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:44.803266 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ed6d2-ab95-40e6-912d-819cbafe36e6 admin admin] PATH_INFO: `/v3/users/cc0344c013424c4fae11efd76a4dcf91` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:44.804580 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ed6d2-ab95-40e6-912d-819cbafe36e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc0344c013424c4fae11efd76a4dcf91)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:44.806950 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ed6d2-ab95-40e6-912d-819cbafe36e6 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:44.824720 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-8a40efa0-1136-4286-bfce-acd4660ac49a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e5bffa3e665482584a466ea1a0dcfa9'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:44.829459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:44.835034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-8a40efa0-1136-4286-bfce-acd4660ac49a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e5bffa3e665482584a466ea1a0dcfa9'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:44.835705 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-8a40efa0-1136-4286-bfce-acd4660ac49a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e5bffa3e665482584a466ea1a0dcfa9'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:44.852193 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-8a40efa0-1136-4286-bfce-acd4660ac49a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e5bffa3e665482584a466ea1a0dcfa9'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:44.852960 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-8a40efa0-1136-4286-bfce-acd4660ac49a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e5bffa3e665482584a466ea1a0dcfa9'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:44.880544 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1062/2121] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:44 2022] DELETE /identity/v3/users/3e5bffa3e665482584a466ea1a0dcfa9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:44.882050 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3aac4a-0434-4a39-91ac-835134b2820d None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:44.882338 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3aac4a-0434-4a39-91ac-835134b2820d None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:44.882615 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3aac4a-0434-4a39-91ac-835134b2820d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:44.886280 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-db2ed6d2-ab95-40e6-912d-819cbafe36e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc0344c013424c4fae11efd76a4dcf91'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:44.897587 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-db2ed6d2-ab95-40e6-912d-819cbafe36e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc0344c013424c4fae11efd76a4dcf91'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:44.898341 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-db2ed6d2-ab95-40e6-912d-819cbafe36e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc0344c013424c4fae11efd76a4dcf91'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:44.906985 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-9e3aac4a-0434-4a39-91ac-835134b2820d None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:44.909183 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-9e3aac4a-0434-4a39-91ac-835134b2820d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:44.913350 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-db2ed6d2-ab95-40e6-912d-819cbafe36e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc0344c013424c4fae11efd76a4dcf91'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:44.927252 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-db2ed6d2-ab95-40e6-912d-819cbafe36e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc0344c013424c4fae11efd76a4dcf91'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:44.941490 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1060/2122] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:44 2022] DELETE /identity/v3/users/cc0344c013424c4fae11efd76a4dcf91 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:44.945009 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fcf5a-f6d3-4fc0-bee1-c743df6940df None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:44.945529 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fcf5a-f6d3-4fc0-bee1-c743df6940df None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:44.945998 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fcf5a-f6d3-4fc0-bee1-c743df6940df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:44.988016 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-d34fcf5a-f6d3-4fc0-bee1-c743df6940df None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:44.993258 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d34fcf5a-f6d3-4fc0-bee1-c743df6940df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:45.017812 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1063/2123] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:45.030436 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3aac4a-0434-4a39-91ac-835134b2820d None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:45.033697 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-9e3aac4a-0434-4a39-91ac-835134b2820d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:45.080049 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:45.121190 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1061/2124] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:45.121736 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3aac4a-0434-4a39-91ac-835134b2820d None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:45.131281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fcf5a-f6d3-4fc0-bee1-c743df6940df None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:45.205837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fcf5a-f6d3-4fc0-bee1-c743df6940df None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:45.211137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671bbde-d485-4cd9-b779-1a0674e3dc2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:45.212062 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671bbde-d485-4cd9-b779-1a0674e3dc2c service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:45.212684 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671bbde-d485-4cd9-b779-1a0674e3dc2c service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:45.213525 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671bbde-d485-4cd9-b779-1a0674e3dc2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:45.225419 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e5ef5-66a1-4b2b-b3fe-b47183c4cb93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:45.226205 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e5ef5-66a1-4b2b-b3fe-b47183c4cb93 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:45.226475 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671bbde-d485-4cd9-b779-1a0674e3dc2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:45.227405 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e5ef5-66a1-4b2b-b3fe-b47183c4cb93 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:45.227791 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e5ef5-66a1-4b2b-b3fe-b47183c4cb93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:45.229044 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671bbde-d485-4cd9-b779-1a0674e3dc2c service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:45.244779 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1062/2125] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:40:45.261974 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e5ef5-66a1-4b2b-b3fe-b47183c4cb93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:45.263521 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e5ef5-66a1-4b2b-b3fe-b47183c4cb93 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:45.288709 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1064/2126] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:40:45.739500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671bbde-d485-4cd9-b779-1a0674e3dc2c service neutron] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:45.741785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-f671bbde-d485-4cd9-b779-1a0674e3dc2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:45.785043 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671bbde-d485-4cd9-b779-1a0674e3dc2c service neutron] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:45.794258 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bc60f-cb3f-4bd5-9e6a-cecccbb92f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:45.795242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bc60f-cb3f-4bd5-9e6a-cecccbb92f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:45.795849 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bc60f-cb3f-4bd5-9e6a-cecccbb92f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:45.796349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bc60f-cb3f-4bd5-9e6a-cecccbb92f48 admin admin] PATH_INFO: `/v3/projects/0135207082434f3f8427a1edd53c3f7f` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:45.798268 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19bc60f-cb3f-4bd5-9e6a-cecccbb92f48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0135207082434f3f8427a1edd53c3f7f)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:45.801262 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19bc60f-cb3f-4bd5-9e6a-cecccbb92f48 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:45.829337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e5ef5-66a1-4b2b-b3fe-b47183c4cb93 service neutron] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:45.830981 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-d25e5ef5-66a1-4b2b-b3fe-b47183c4cb93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:45.874023 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e5ef5-66a1-4b2b-b3fe-b47183c4cb93 service neutron] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:45.885716 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb25366-04ef-4291-b8e8-1e3dc3e9a5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:45.887228 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb25366-04ef-4291-b8e8-1e3dc3e9a5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:45.888114 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb25366-04ef-4291-b8e8-1e3dc3e9a5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:45.888886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb25366-04ef-4291-b8e8-1e3dc3e9a5d8 admin admin] PATH_INFO: `/v3/projects/bdf3423289504a9191c6ea15be9bb6de` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:45.889472 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:45.893914 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb25366-04ef-4291-b8e8-1e3dc3e9a5d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdf3423289504a9191c6ea15be9bb6de)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:45.899974 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb25366-04ef-4291-b8e8-1e3dc3e9a5d8 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:45.920981 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f19bc60f-cb3f-4bd5-9e6a-cecccbb92f48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0135207082434f3f8427a1edd53c3f7f', 'request_id': 'req-f19bc60f-cb3f-4bd5-9e6a-cecccbb92f48'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:45.941869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f19bc60f-cb3f-4bd5-9e6a-cecccbb92f48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0135207082434f3f8427a1edd53c3f7f', 'request_id': 'req-f19bc60f-cb3f-4bd5-9e6a-cecccbb92f48'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:45.952571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f19bc60f-cb3f-4bd5-9e6a-cecccbb92f48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0135207082434f3f8427a1edd53c3f7f', 'request_id': 'req-f19bc60f-cb3f-4bd5-9e6a-cecccbb92f48'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:45.973093 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f19bc60f-cb3f-4bd5-9e6a-cecccbb92f48 admin admin] The token cache is being invalidate because project 0135207082434f3f8427a1edd53c3f7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:45.973378 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f19bc60f-cb3f-4bd5-9e6a-cecccbb92f48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:45.974330 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f19bc60f-cb3f-4bd5-9e6a-cecccbb92f48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:45.976586 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1063/2127] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:45 2022] DELETE /identity/v3/projects/0135207082434f3f8427a1edd53c3f7f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:45.982556 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bc60f-cb3f-4bd5-9e6a-cecccbb92f48 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:45.984962 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-f19bc60f-cb3f-4bd5-9e6a-cecccbb92f48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:46.001921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2fb25366-04ef-4291-b8e8-1e3dc3e9a5d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdf3423289504a9191c6ea15be9bb6de', 'request_id': 'req-2fb25366-04ef-4291-b8e8-1e3dc3e9a5d8'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.021202 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2fb25366-04ef-4291-b8e8-1e3dc3e9a5d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdf3423289504a9191c6ea15be9bb6de', 'request_id': 'req-2fb25366-04ef-4291-b8e8-1e3dc3e9a5d8'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.033096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2fb25366-04ef-4291-b8e8-1e3dc3e9a5d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdf3423289504a9191c6ea15be9bb6de', 'request_id': 'req-2fb25366-04ef-4291-b8e8-1e3dc3e9a5d8'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.052967 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2fb25366-04ef-4291-b8e8-1e3dc3e9a5d8 admin admin] The token cache is being invalidate because project bdf3423289504a9191c6ea15be9bb6de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:46.053363 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2fb25366-04ef-4291-b8e8-1e3dc3e9a5d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.054113 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-2fb25366-04ef-4291-b8e8-1e3dc3e9a5d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.056014 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1065/2128] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:45 2022] DELETE /identity/v3/projects/bdf3423289504a9191c6ea15be9bb6de => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:46.060593 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb25366-04ef-4291-b8e8-1e3dc3e9a5d8 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:46.062817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-2fb25366-04ef-4291-b8e8-1e3dc3e9a5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:46.123071 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bc60f-cb3f-4bd5-9e6a-cecccbb92f48 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:46.146916 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb25366-04ef-4291-b8e8-1e3dc3e9a5d8 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:46.151936 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3116ac-7968-4f44-ba45-b71e2a03ba7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:46.152881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3116ac-7968-4f44-ba45-b71e2a03ba7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:46.153283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3116ac-7968-4f44-ba45-b71e2a03ba7d admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:46.153628 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3116ac-7968-4f44-ba45-b71e2a03ba7d admin admin] PATH_INFO: `/v3/users/5c60b3e3a24046a8ad1343ad64c57f1b` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:46.154923 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3116ac-7968-4f44-ba45-b71e2a03ba7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c60b3e3a24046a8ad1343ad64c57f1b)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:46.156918 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3116ac-7968-4f44-ba45-b71e2a03ba7d admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:46.163710 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:46.188806 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249167f5-f05d-4427-986c-f044aaf0bc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:46.189731 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249167f5-f05d-4427-986c-f044aaf0bc82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:46.190021 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249167f5-f05d-4427-986c-f044aaf0bc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:46.190291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249167f5-f05d-4427-986c-f044aaf0bc82 admin admin] PATH_INFO: `/v3/users/446cca77db594a3f9d6e8e76d33fb63f` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:46.192757 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249167f5-f05d-4427-986c-f044aaf0bc82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=446cca77db594a3f9d6e8e76d33fb63f)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:46.195612 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249167f5-f05d-4427-986c-f044aaf0bc82 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:46.224394 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1c3116ac-7968-4f44-ba45-b71e2a03ba7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c60b3e3a24046a8ad1343ad64c57f1b'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.236935 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1c3116ac-7968-4f44-ba45-b71e2a03ba7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c60b3e3a24046a8ad1343ad64c57f1b'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.237620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1c3116ac-7968-4f44-ba45-b71e2a03ba7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c60b3e3a24046a8ad1343ad64c57f1b'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.247333 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1c3116ac-7968-4f44-ba45-b71e2a03ba7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c60b3e3a24046a8ad1343ad64c57f1b'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.248009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1c3116ac-7968-4f44-ba45-b71e2a03ba7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c60b3e3a24046a8ad1343ad64c57f1b'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.263185 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-249167f5-f05d-4427-986c-f044aaf0bc82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '446cca77db594a3f9d6e8e76d33fb63f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.268949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1066/2129] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:46 2022] DELETE /identity/v3/users/5c60b3e3a24046a8ad1343ad64c57f1b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:46.276793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-249167f5-f05d-4427-986c-f044aaf0bc82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '446cca77db594a3f9d6e8e76d33fb63f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.282732 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-249167f5-f05d-4427-986c-f044aaf0bc82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '446cca77db594a3f9d6e8e76d33fb63f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.292042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:46.303320 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-249167f5-f05d-4427-986c-f044aaf0bc82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '446cca77db594a3f9d6e8e76d33fb63f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.326513 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-249167f5-f05d-4427-986c-f044aaf0bc82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '446cca77db594a3f9d6e8e76d33fb63f'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.347845 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1064/2130] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:45 2022] DELETE /identity/v3/users/446cca77db594a3f9d6e8e76d33fb63f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:46.671494 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3116ac-7968-4f44-ba45-b71e2a03ba7d admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:46.673399 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-1c3116ac-7968-4f44-ba45-b71e2a03ba7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:46.738009 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249167f5-f05d-4427-986c-f044aaf0bc82 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:46.740046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-249167f5-f05d-4427-986c-f044aaf0bc82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:46.753017 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3116ac-7968-4f44-ba45-b71e2a03ba7d admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:46.814878 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e1dc6-6b38-4fc3-a1ff-342a8d985866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:46.815750 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e1dc6-6b38-4fc3-a1ff-342a8d985866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:46.815882 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249167f5-f05d-4427-986c-f044aaf0bc82 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:46.816025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e1dc6-6b38-4fc3-a1ff-342a8d985866 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:46.816289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e1dc6-6b38-4fc3-a1ff-342a8d985866 admin admin] PATH_INFO: `/v3/projects/f4b5aefc436446ac99ff3dcd154bf2b5` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:46.817513 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e1dc6-6b38-4fc3-a1ff-342a8d985866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b5aefc436446ac99ff3dcd154bf2b5)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:46.819432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e1dc6-6b38-4fc3-a1ff-342a8d985866 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:46.820591 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76467f4b-5dc2-4d86-bb57-738bee98f095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:46.821498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76467f4b-5dc2-4d86-bb57-738bee98f095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:46.821788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76467f4b-5dc2-4d86-bb57-738bee98f095 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:46.822066 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76467f4b-5dc2-4d86-bb57-738bee98f095 admin admin] PATH_INFO: `/v3/projects/960de2ba31c1488cacc1c6312b8b6c36` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:46.823899 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76467f4b-5dc2-4d86-bb57-738bee98f095 admin admin] RBAC: Authorizing `identity:delete_project(project_id=960de2ba31c1488cacc1c6312b8b6c36)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:46.825865 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76467f4b-5dc2-4d86-bb57-738bee98f095 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:46.861122 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:46.873777 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-707e1dc6-6b38-4fc3-a1ff-342a8d985866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b5aefc436446ac99ff3dcd154bf2b5', 'request_id': 'req-707e1dc6-6b38-4fc3-a1ff-342a8d985866'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.874873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-76467f4b-5dc2-4d86-bb57-738bee98f095 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '960de2ba31c1488cacc1c6312b8b6c36', 'request_id': 'req-76467f4b-5dc2-4d86-bb57-738bee98f095'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.881879 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-707e1dc6-6b38-4fc3-a1ff-342a8d985866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b5aefc436446ac99ff3dcd154bf2b5', 'request_id': 'req-707e1dc6-6b38-4fc3-a1ff-342a8d985866'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.883782 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-76467f4b-5dc2-4d86-bb57-738bee98f095 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '960de2ba31c1488cacc1c6312b8b6c36', 'request_id': 'req-76467f4b-5dc2-4d86-bb57-738bee98f095'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.888937 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-707e1dc6-6b38-4fc3-a1ff-342a8d985866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b5aefc436446ac99ff3dcd154bf2b5', 'request_id': 'req-707e1dc6-6b38-4fc3-a1ff-342a8d985866'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.890744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-76467f4b-5dc2-4d86-bb57-738bee98f095 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '960de2ba31c1488cacc1c6312b8b6c36', 'request_id': 'req-76467f4b-5dc2-4d86-bb57-738bee98f095'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.900809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-707e1dc6-6b38-4fc3-a1ff-342a8d985866 admin admin] The token cache is being invalidate because project f4b5aefc436446ac99ff3dcd154bf2b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:46.901120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-707e1dc6-6b38-4fc3-a1ff-342a8d985866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.901774 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-707e1dc6-6b38-4fc3-a1ff-342a8d985866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.902353 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-76467f4b-5dc2-4d86-bb57-738bee98f095 admin admin] The token cache is being invalidate because project 960de2ba31c1488cacc1c6312b8b6c36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:46.902644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-76467f4b-5dc2-4d86-bb57-738bee98f095 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.903512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1067/2131] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:46 2022] DELETE /identity/v3/projects/f4b5aefc436446ac99ff3dcd154bf2b5 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:46.903638 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-76467f4b-5dc2-4d86-bb57-738bee98f095 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:46.905142 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1065/2132] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:46 2022] DELETE /identity/v3/projects/960de2ba31c1488cacc1c6312b8b6c36 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:46.908074 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03c673-547d-43aa-85c9-11c56c03f9fd None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:46.908368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03c673-547d-43aa-85c9-11c56c03f9fd None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:46.908647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03c673-547d-43aa-85c9-11c56c03f9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:46.908756 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76467f4b-5dc2-4d86-bb57-738bee98f095 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:46.910140 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-76467f4b-5dc2-4d86-bb57-738bee98f095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:46.927434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-0b03c673-547d-43aa-85c9-11c56c03f9fd None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:46.928642 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-0b03c673-547d-43aa-85c9-11c56c03f9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:46.976695 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76467f4b-5dc2-4d86-bb57-738bee98f095 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:46.982386 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5acd78-8b60-4ddf-95e0-275b46629cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:46.983271 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5acd78-8b60-4ddf-95e0-275b46629cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:46.983548 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5acd78-8b60-4ddf-95e0-275b46629cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:46.983832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5acd78-8b60-4ddf-95e0-275b46629cc6 admin admin] PATH_INFO: `/v3/users/09897d75f24843d185a7cdde0e3fedc6` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:46.985196 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5acd78-8b60-4ddf-95e0-275b46629cc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09897d75f24843d185a7cdde0e3fedc6)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:46.987009 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5acd78-8b60-4ddf-95e0-275b46629cc6 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:46.992136 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1068/2133] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:47.000036 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:47.000199 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d3df2-0f9b-4898-b771-fb0b7b835a02 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.000328 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d3df2-0f9b-4898-b771-fb0b7b835a02 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.000611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d3df2-0f9b-4898-b771-fb0b7b835a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.022121 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-445d3df2-0f9b-4898-b771-fb0b7b835a02 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:47.023844 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-445d3df2-0f9b-4898-b771-fb0b7b835a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:47.032732 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1069/2134] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:47.036837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d3df2-0f9b-4898-b771-fb0b7b835a02 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:47.039603 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-ae5acd78-8b60-4ddf-95e0-275b46629cc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09897d75f24843d185a7cdde0e3fedc6'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:47.047816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d3df2-0f9b-4898-b771-fb0b7b835a02 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:47.049609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-ae5acd78-8b60-4ddf-95e0-275b46629cc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09897d75f24843d185a7cdde0e3fedc6'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:47.050322 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-ae5acd78-8b60-4ddf-95e0-275b46629cc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09897d75f24843d185a7cdde0e3fedc6'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:47.053494 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c50f6-18d4-4a92-8995-ee6fb1b8ef90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:47.054248 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c50f6-18d4-4a92-8995-ee6fb1b8ef90 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.054539 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c50f6-18d4-4a92-8995-ee6fb1b8ef90 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.054818 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c50f6-18d4-4a92-8995-ee6fb1b8ef90 admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.055560 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c50f6-18d4-4a92-8995-ee6fb1b8ef90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:47.055875 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c50f6-18d4-4a92-8995-ee6fb1b8ef90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:47.057519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c50f6-18d4-4a92-8995-ee6fb1b8ef90 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:47.069581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-ae5acd78-8b60-4ddf-95e0-275b46629cc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09897d75f24843d185a7cdde0e3fedc6'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:47.071438 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1070/2135] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:40:47.077343 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c50f6-18d4-4a92-8995-ee6fb1b8ef90 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:47.080631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-b09c50f6-18d4-4a92-8995-ee6fb1b8ef90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:47.083938 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-ae5acd78-8b60-4ddf-95e0-275b46629cc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09897d75f24843d185a7cdde0e3fedc6'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:47.094341 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1066/2136] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:46 2022] DELETE /identity/v3/users/09897d75f24843d185a7cdde0e3fedc6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:47.095287 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c50f6-18d4-4a92-8995-ee6fb1b8ef90 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:47.180722 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f33ec-e90b-41c9-99c1-7aa2e0077a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:47.182268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f33ec-e90b-41c9-99c1-7aa2e0077a12 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.182714 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f33ec-e90b-41c9-99c1-7aa2e0077a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.183153 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f33ec-e90b-41c9-99c1-7aa2e0077a12 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.184289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f33ec-e90b-41c9-99c1-7aa2e0077a12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:47.186833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f33ec-e90b-41c9-99c1-7aa2e0077a12 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:47.221957 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1071/2137] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:47 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:40:47.226352 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5acd78-8b60-4ddf-95e0-275b46629cc6 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:47.240801 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5acd78-8b60-4ddf-95e0-275b46629cc6 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:47.338261 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66959089-51a2-4697-9d29-126a9071db34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:47.338948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66959089-51a2-4697-9d29-126a9071db34 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.339768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66959089-51a2-4697-9d29-126a9071db34 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.340235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66959089-51a2-4697-9d29-126a9071db34 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.341026 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66959089-51a2-4697-9d29-126a9071db34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:47.343281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66959089-51a2-4697-9d29-126a9071db34 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:47.372300 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1067/2138] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:47 2022] POST /identity/v3/users => generated 291 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:47.374246 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f33ec-e90b-41c9-99c1-7aa2e0077a12 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:47.386558 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f33ec-e90b-41c9-99c1-7aa2e0077a12 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:47.392030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2680564-f406-448a-979f-bd1d100640bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:47.392704 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2680564-f406-448a-979f-bd1d100640bb admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.392989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2680564-f406-448a-979f-bd1d100640bb admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.393312 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2680564-f406-448a-979f-bd1d100640bb admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.394005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2680564-f406-448a-979f-bd1d100640bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:47.395621 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2680564-f406-448a-979f-bd1d100640bb admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:47.419218 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1072/2139] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:47.423327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66959089-51a2-4697-9d29-126a9071db34 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:47.436661 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66959089-51a2-4697-9d29-126a9071db34 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:47.459283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af950b78-96ed-4f2a-b5ae-e84a8c320ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:47.460179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af950b78-96ed-4f2a-b5ae-e84a8c320ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.460544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af950b78-96ed-4f2a-b5ae-e84a8c320ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.460893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af950b78-96ed-4f2a-b5ae-e84a8c320ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.461763 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af950b78-96ed-4f2a-b5ae-e84a8c320ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:47.463565 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af950b78-96ed-4f2a-b5ae-e84a8c320ea8 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:47.481713 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1068/2140] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:47.486016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2680564-f406-448a-979f-bd1d100640bb admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:47.498036 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2680564-f406-448a-979f-bd1d100640bb admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:47.504431 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f68963-c9c5-44b4-a34a-eca709e1966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:47.505451 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f68963-c9c5-44b4-a34a-eca709e1966f admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.505958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f68963-c9c5-44b4-a34a-eca709e1966f admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.506442 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f68963-c9c5-44b4-a34a-eca709e1966f admin admin] PATH_INFO: `/v3/projects/8a3318d9534c4527b6d3d4035239e868/users/83dc0f6719b8412ea6a46e22397841c1/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.528038 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f68963-c9c5-44b4-a34a-eca709e1966f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a3318d9534c4527b6d3d4035239e868, user_id=83dc0f6719b8412ea6a46e22397841c1, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:47.537015 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f68963-c9c5-44b4-a34a-eca709e1966f admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:47.540029 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:47.558846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1073/2141] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:47 2022] PUT /identity/v3/projects/8a3318d9534c4527b6d3d4035239e868/users/83dc0f6719b8412ea6a46e22397841c1/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:47.573024 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d35d57-2d8c-4245-bc06-8ae6ecdec9e6 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.573496 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d35d57-2d8c-4245-bc06-8ae6ecdec9e6 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.573851 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d35d57-2d8c-4245-bc06-8ae6ecdec9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.631546 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-17d35d57-2d8c-4245-bc06-8ae6ecdec9e6 None None] MFA Rules not processed for user `83dc0f6719b8412ea6a46e22397841c1`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:47.632825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-17d35d57-2d8c-4245-bc06-8ae6ecdec9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:47.646431 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f68963-c9c5-44b4-a34a-eca709e1966f admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:47.649058 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-54f68963-c9c5-44b4-a34a-eca709e1966f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:47.735356 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f68963-c9c5-44b4-a34a-eca709e1966f admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:47.740856 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b76538-393e-47e8-be8c-e0bf9cb7c545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:47.741784 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b76538-393e-47e8-be8c-e0bf9cb7c545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.742090 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b76538-393e-47e8-be8c-e0bf9cb7c545 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.742412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b76538-393e-47e8-be8c-e0bf9cb7c545 admin admin] PATH_INFO: `/v3/projects/62b499ef959d4109913f52de01f30a59` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.743558 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b76538-393e-47e8-be8c-e0bf9cb7c545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62b499ef959d4109913f52de01f30a59)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:47.746093 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b76538-393e-47e8-be8c-e0bf9cb7c545 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:47.753096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:47.759187 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1069/2142] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:47.764508 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d35d57-2d8c-4245-bc06-8ae6ecdec9e6 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:47.777996 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d35d57-2d8c-4245-bc06-8ae6ecdec9e6 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:47.784034 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da7b9c-9e77-49cf-bff6-c4ef637c4f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:47.784705 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da7b9c-9e77-49cf-bff6-c4ef637c4f4c admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.784999 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da7b9c-9e77-49cf-bff6-c4ef637c4f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.785358 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da7b9c-9e77-49cf-bff6-c4ef637c4f4c admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.786029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da7b9c-9e77-49cf-bff6-c4ef637c4f4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:47.788048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da7b9c-9e77-49cf-bff6-c4ef637c4f4c admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:47.809327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-35b76538-393e-47e8-be8c-e0bf9cb7c545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62b499ef959d4109913f52de01f30a59', 'request_id': 'req-35b76538-393e-47e8-be8c-e0bf9cb7c545'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:47.818710 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-35b76538-393e-47e8-be8c-e0bf9cb7c545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62b499ef959d4109913f52de01f30a59', 'request_id': 'req-35b76538-393e-47e8-be8c-e0bf9cb7c545'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:47.825785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1070/2143] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:47 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:47.826288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-35b76538-393e-47e8-be8c-e0bf9cb7c545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62b499ef959d4109913f52de01f30a59', 'request_id': 'req-35b76538-393e-47e8-be8c-e0bf9cb7c545'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:47.829509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da7b9c-9e77-49cf-bff6-c4ef637c4f4c admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:47.838380 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-35b76538-393e-47e8-be8c-e0bf9cb7c545 admin admin] The token cache is being invalidate because project 62b499ef959d4109913f52de01f30a59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:47.838667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-35b76538-393e-47e8-be8c-e0bf9cb7c545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:47.839337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-35b76538-393e-47e8-be8c-e0bf9cb7c545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:47.841300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1074/2144] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:47 2022] DELETE /identity/v3/projects/62b499ef959d4109913f52de01f30a59 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:47.846386 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defe446-4a63-43ce-9e21-15e003de5b5e None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.846673 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defe446-4a63-43ce-9e21-15e003de5b5e None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.846949 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defe446-4a63-43ce-9e21-15e003de5b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.866867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-4defe446-4a63-43ce-9e21-15e003de5b5e None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:47.868950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-4defe446-4a63-43ce-9e21-15e003de5b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:47.911575 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da7b9c-9e77-49cf-bff6-c4ef637c4f4c admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:47.917371 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79ea4b-aaab-4a34-8597-a60945cf4dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:47.918156 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79ea4b-aaab-4a34-8597-a60945cf4dbc admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.918500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79ea4b-aaab-4a34-8597-a60945cf4dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.918977 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79ea4b-aaab-4a34-8597-a60945cf4dbc admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.919840 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79ea4b-aaab-4a34-8597-a60945cf4dbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:47.921835 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79ea4b-aaab-4a34-8597-a60945cf4dbc admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:47.946489 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1071/2145] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:47 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:47.946719 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1075/2146] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:47.952365 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79ea4b-aaab-4a34-8597-a60945cf4dbc admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:47.953282 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155cae6-1f10-4b46-ada6-c74b2c3ca920 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.953588 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155cae6-1f10-4b46-ada6-c74b2c3ca920 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.953968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155cae6-1f10-4b46-ada6-c74b2c3ca920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.954209 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-0b79ea4b-aaab-4a34-8597-a60945cf4dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:47.968444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79ea4b-aaab-4a34-8597-a60945cf4dbc admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:47.974799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224e516-1ab1-4503-8dd2-9cd7ac333dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:47.975481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224e516-1ab1-4503-8dd2-9cd7ac333dfb admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.975824 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224e516-1ab1-4503-8dd2-9cd7ac333dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.976135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224e516-1ab1-4503-8dd2-9cd7ac333dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.976940 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224e516-1ab1-4503-8dd2-9cd7ac333dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:47.978717 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224e516-1ab1-4503-8dd2-9cd7ac333dfb admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:47.979693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-4155cae6-1f10-4b46-ada6-c74b2c3ca920 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:47.982143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-4155cae6-1f10-4b46-ada6-c74b2c3ca920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:47.993259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1072/2147] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:47.994569 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224e516-1ab1-4503-8dd2-9cd7ac333dfb admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:47.995388 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1076/2148] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:47.999089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155cae6-1f10-4b46-ada6-c74b2c3ca920 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:48.005499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224e516-1ab1-4503-8dd2-9cd7ac333dfb admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:48.011110 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155cae6-1f10-4b46-ada6-c74b2c3ca920 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:48.011656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37dcad-56ce-46a1-b817-58d2e276bd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:48.012723 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37dcad-56ce-46a1-b817-58d2e276bd8a admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:48.013015 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37dcad-56ce-46a1-b817-58d2e276bd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:48.013342 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37dcad-56ce-46a1-b817-58d2e276bd8a admin admin] PATH_INFO: `/v3/projects/ba76ffba63b948fb97f5d040fc41a0ce/users/972582cfce4a4d098b5c0137208c9b7e/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:48.017498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680f790-a8a2-4e20-baee-e9b631ca4c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:48.018836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680f790-a8a2-4e20-baee-e9b631ca4c60 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:48.019223 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680f790-a8a2-4e20-baee-e9b631ca4c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:48.019604 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680f790-a8a2-4e20-baee-e9b631ca4c60 admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:48.020465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680f790-a8a2-4e20-baee-e9b631ca4c60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:48.020885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680f790-a8a2-4e20-baee-e9b631ca4c60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:48.022810 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680f790-a8a2-4e20-baee-e9b631ca4c60 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:48.031746 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37dcad-56ce-46a1-b817-58d2e276bd8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba76ffba63b948fb97f5d040fc41a0ce, user_id=972582cfce4a4d098b5c0137208c9b7e, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:48.034513 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37dcad-56ce-46a1-b817-58d2e276bd8a admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:48.038873 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1077/2149] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:40:48.044145 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680f790-a8a2-4e20-baee-e9b631ca4c60 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:48.051407 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:48.052041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1073/2150] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:47 2022] PUT /identity/v3/projects/ba76ffba63b948fb97f5d040fc41a0ce/users/972582cfce4a4d098b5c0137208c9b7e/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:48.055698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37dcad-56ce-46a1-b817-58d2e276bd8a admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:48.059506 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680f790-a8a2-4e20-baee-e9b631ca4c60 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:48.129596 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37dcad-56ce-46a1-b817-58d2e276bd8a admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:48.134476 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced166f-a623-4716-ad0d-11b3c51ca092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:48.135934 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced166f-a623-4716-ad0d-11b3c51ca092 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:48.136586 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4d420-b461-40e0-9c20-55a6eb092a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:48.136717 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced166f-a623-4716-ad0d-11b3c51ca092 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:48.136717 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced166f-a623-4716-ad0d-11b3c51ca092 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:48.137027 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4d420-b461-40e0-9c20-55a6eb092a2c admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:48.137438 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4d420-b461-40e0-9c20-55a6eb092a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:48.137800 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced166f-a623-4716-ad0d-11b3c51ca092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:48.138140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4d420-b461-40e0-9c20-55a6eb092a2c admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:48.139063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4d420-b461-40e0-9c20-55a6eb092a2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:48.140110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced166f-a623-4716-ad0d-11b3c51ca092 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:48.141060 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4d420-b461-40e0-9c20-55a6eb092a2c admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:48.152347 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1074/2151] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:40:48.156572 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced166f-a623-4716-ad0d-11b3c51ca092 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:48.168288 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced166f-a623-4716-ad0d-11b3c51ca092 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:48.175685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1078/2152] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:48 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:48.176727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbca90-48e1-4c4f-9108-113db34fdef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:48.176886 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4d420-b461-40e0-9c20-55a6eb092a2c admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:48.177857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbca90-48e1-4c4f-9108-113db34fdef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:48.178244 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbca90-48e1-4c4f-9108-113db34fdef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:48.178625 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbca90-48e1-4c4f-9108-113db34fdef7 admin admin] PATH_INFO: `/v3/projects/ba76ffba63b948fb97f5d040fc41a0ce/users/972582cfce4a4d098b5c0137208c9b7e/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:48.180896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbca90-48e1-4c4f-9108-113db34fdef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba76ffba63b948fb97f5d040fc41a0ce, user_id=972582cfce4a4d098b5c0137208c9b7e, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:48.184217 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbca90-48e1-4c4f-9108-113db34fdef7 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:48.198658 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1075/2153] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:48 2022] PUT /identity/v3/projects/ba76ffba63b948fb97f5d040fc41a0ce/users/972582cfce4a4d098b5c0137208c9b7e/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:48.208902 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd82a3-3288-4540-b6da-074d7c999648 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:48.209229 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd82a3-3288-4540-b6da-074d7c999648 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:48.209516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd82a3-3288-4540-b6da-074d7c999648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:48.244944 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:48.248788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-22cd82a3-3288-4540-b6da-074d7c999648 None None] MFA Rules not processed for user `972582cfce4a4d098b5c0137208c9b7e`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:48.251531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-22cd82a3-3288-4540-b6da-074d7c999648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:48.255796 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4d420-b461-40e0-9c20-55a6eb092a2c admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:48.324480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147e27f-f665-4a14-a415-adaaef8f78df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:48.325275 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147e27f-f665-4a14-a415-adaaef8f78df admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:48.325646 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147e27f-f665-4a14-a415-adaaef8f78df admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:48.325986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147e27f-f665-4a14-a415-adaaef8f78df admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:48.327047 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147e27f-f665-4a14-a415-adaaef8f78df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:48.329397 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147e27f-f665-4a14-a415-adaaef8f78df admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:48.331717 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1076/2154] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:48.344617 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd82a3-3288-4540-b6da-074d7c999648 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:48.346446 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-22cd82a3-3288-4540-b6da-074d7c999648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:48.359654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1079/2155] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:48 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:40:48.360693 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147e27f-f665-4a14-a415-adaaef8f78df admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:48.375435 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147e27f-f665-4a14-a415-adaaef8f78df admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:48.380353 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515607b5-366d-460c-bbba-341109b5c1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:48.381019 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515607b5-366d-460c-bbba-341109b5c1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:48.381335 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515607b5-366d-460c-bbba-341109b5c1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:48.381629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515607b5-366d-460c-bbba-341109b5c1d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:48.382483 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515607b5-366d-460c-bbba-341109b5c1d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:48.384120 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515607b5-366d-460c-bbba-341109b5c1d5 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:48.394926 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1080/2156] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:40:48.399495 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515607b5-366d-460c-bbba-341109b5c1d5 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:48.410068 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515607b5-366d-460c-bbba-341109b5c1d5 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:48.411132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd82a3-3288-4540-b6da-074d7c999648 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:48.415954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c856c-2722-42af-afab-0b6a186f4f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:48.416751 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c856c-2722-42af-afab-0b6a186f4f70 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:48.416924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a24e4-31ae-4a1c-b251-1275f8e49a54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:48.417210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c856c-2722-42af-afab-0b6a186f4f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:48.417563 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c856c-2722-42af-afab-0b6a186f4f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:48.417803 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a24e4-31ae-4a1c-b251-1275f8e49a54 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:48.418152 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a24e4-31ae-4a1c-b251-1275f8e49a54 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:48.418502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a24e4-31ae-4a1c-b251-1275f8e49a54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:48.418606 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c856c-2722-42af-afab-0b6a186f4f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:48.420247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c856c-2722-42af-afab-0b6a186f4f70 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:48.429377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a24e4-31ae-4a1c-b251-1275f8e49a54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:48.430874 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a24e4-31ae-4a1c-b251-1275f8e49a54 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:48.432344 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1081/2157] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:48.436664 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c856c-2722-42af-afab-0b6a186f4f70 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:48.442395 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1077/2158] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:40:48.445346 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:48.450797 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c856c-2722-42af-afab-0b6a186f4f70 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:48.456852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449475d0-2b0d-42da-82e1-db4f92f9ed5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:48.459099 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449475d0-2b0d-42da-82e1-db4f92f9ed5f admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:48.460614 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449475d0-2b0d-42da-82e1-db4f92f9ed5f admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:48.461906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449475d0-2b0d-42da-82e1-db4f92f9ed5f admin admin] PATH_INFO: `/v3/projects/91f207a645a147e2b74c526a1bb45260/users/24306a0e1b3c4ae08576a6b6762af0be/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:48.488107 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449475d0-2b0d-42da-82e1-db4f92f9ed5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f207a645a147e2b74c526a1bb45260, user_id=24306a0e1b3c4ae08576a6b6762af0be, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:48.492181 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449475d0-2b0d-42da-82e1-db4f92f9ed5f admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:48.510587 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1082/2159] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:48 2022] PUT /identity/v3/projects/91f207a645a147e2b74c526a1bb45260/users/24306a0e1b3c4ae08576a6b6762af0be/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:48.515688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8fcb6-4765-448c-ad0b-4bc42ff4038c None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:48.515982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8fcb6-4765-448c-ad0b-4bc42ff4038c None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:48.516231 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8fcb6-4765-448c-ad0b-4bc42ff4038c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:48.536428 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-d6d8fcb6-4765-448c-ad0b-4bc42ff4038c None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:48.537726 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d6d8fcb6-4765-448c-ad0b-4bc42ff4038c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:48.566814 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449475d0-2b0d-42da-82e1-db4f92f9ed5f admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:48.618604 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1078/2160] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:48.630772 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8fcb6-4765-448c-ad0b-4bc42ff4038c None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:48.634098 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d6d8fcb6-4765-448c-ad0b-4bc42ff4038c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:48.663215 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449475d0-2b0d-42da-82e1-db4f92f9ed5f admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:48.668473 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a968a-2cb0-4df2-a5fa-1b48a45dc59b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:48.669389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a968a-2cb0-4df2-a5fa-1b48a45dc59b service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:48.669745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a968a-2cb0-4df2-a5fa-1b48a45dc59b service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:48.670094 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a968a-2cb0-4df2-a5fa-1b48a45dc59b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:48.672232 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-7e7a968a-2cb0-4df2-a5fa-1b48a45dc59b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:48.751180 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:48.767949 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8fcb6-4765-448c-ad0b-4bc42ff4038c None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:48.772061 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5a949-73f9-407d-b678-accc1aeadff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:48.772793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5a949-73f9-407d-b678-accc1aeadff7 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:48.773576 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5a949-73f9-407d-b678-accc1aeadff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:48.773938 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5a949-73f9-407d-b678-accc1aeadff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:48.787406 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a968a-2cb0-4df2-a5fa-1b48a45dc59b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:48.789053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a968a-2cb0-4df2-a5fa-1b48a45dc59b service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:48.791785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f5a949-73f9-407d-b678-accc1aeadff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:48.794124 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f5a949-73f9-407d-b678-accc1aeadff7 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:48.805478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1079/2161] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:48.811147 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1083/2162] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:49.150630 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5a949-73f9-407d-b678-accc1aeadff7 service neutron] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:49.160714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5a949-73f9-407d-b678-accc1aeadff7 service neutron] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:49.166117 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239316a-48e4-470c-ba51-6eaf88bf6081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:49.167141 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239316a-48e4-470c-ba51-6eaf88bf6081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:49.167620 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239316a-48e4-470c-ba51-6eaf88bf6081 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:49.168083 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239316a-48e4-470c-ba51-6eaf88bf6081 admin admin] PATH_INFO: `/v3/users/83dc0f6719b8412ea6a46e22397841c1` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:49.169806 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239316a-48e4-470c-ba51-6eaf88bf6081 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83dc0f6719b8412ea6a46e22397841c1)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:49.172268 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239316a-48e4-470c-ba51-6eaf88bf6081 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:49.238409 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-8239316a-48e4-470c-ba51-6eaf88bf6081 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83dc0f6719b8412ea6a46e22397841c1'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:49.248770 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-8239316a-48e4-470c-ba51-6eaf88bf6081 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83dc0f6719b8412ea6a46e22397841c1'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:49.261893 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-8239316a-48e4-470c-ba51-6eaf88bf6081 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83dc0f6719b8412ea6a46e22397841c1'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:49.281274 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-8239316a-48e4-470c-ba51-6eaf88bf6081 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83dc0f6719b8412ea6a46e22397841c1'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:49.288799 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-8239316a-48e4-470c-ba51-6eaf88bf6081 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83dc0f6719b8412ea6a46e22397841c1'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:49.296405 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1080/2163] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:49 2022] DELETE /identity/v3/users/83dc0f6719b8412ea6a46e22397841c1 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:49.300896 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0119b31-3216-49ec-b69e-85f603f5dc7c None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:49.301461 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0119b31-3216-49ec-b69e-85f603f5dc7c None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:49.301564 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0119b31-3216-49ec-b69e-85f603f5dc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:49.349405 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-f0119b31-3216-49ec-b69e-85f603f5dc7c None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:49.351178 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f0119b31-3216-49ec-b69e-85f603f5dc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:49.426502 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1084/2164] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:49.437267 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239316a-48e4-470c-ba51-6eaf88bf6081 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:49.439126 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-8239316a-48e4-470c-ba51-6eaf88bf6081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:49.514791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239316a-48e4-470c-ba51-6eaf88bf6081 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:49.520569 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ca774-ba29-4d3d-8a01-17597a83f170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:49.520817 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ca774-ba29-4d3d-8a01-17597a83f170 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:49.521241 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ca774-ba29-4d3d-8a01-17597a83f170 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:49.521643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ca774-ba29-4d3d-8a01-17597a83f170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:49.532398 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ca774-ba29-4d3d-8a01-17597a83f170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:49.534301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ca774-ba29-4d3d-8a01-17597a83f170 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:49.546891 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1081/2165] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:49.891754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0119b31-3216-49ec-b69e-85f603f5dc7c None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:49.893523 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f0119b31-3216-49ec-b69e-85f603f5dc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:49.921512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0119b31-3216-49ec-b69e-85f603f5dc7c None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:49.930481 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813694c8-5085-44d9-9385-444f547751e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:49.932834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813694c8-5085-44d9-9385-444f547751e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:49.933406 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813694c8-5085-44d9-9385-444f547751e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:49.933905 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813694c8-5085-44d9-9385-444f547751e4 admin admin] PATH_INFO: `/v3/projects/8a3318d9534c4527b6d3d4035239e868` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:49.935661 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813694c8-5085-44d9-9385-444f547751e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a3318d9534c4527b6d3d4035239e868)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:49.958729 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813694c8-5085-44d9-9385-444f547751e4 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:50.027825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125f3ac-8149-4833-bbb4-fd27246b8933 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:50.028431 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125f3ac-8149-4833-bbb4-fd27246b8933 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:50.028789 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125f3ac-8149-4833-bbb4-fd27246b8933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:50.087566 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:50.122125 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-813694c8-5085-44d9-9385-444f547751e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a3318d9534c4527b6d3d4035239e868', 'request_id': 'req-813694c8-5085-44d9-9385-444f547751e4'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:50.130665 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-2125f3ac-8149-4833-bbb4-fd27246b8933 None None] MFA Rules not processed for user `24306a0e1b3c4ae08576a6b6762af0be`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:50.134648 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-2125f3ac-8149-4833-bbb4-fd27246b8933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:50.137604 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-813694c8-5085-44d9-9385-444f547751e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a3318d9534c4527b6d3d4035239e868', 'request_id': 'req-813694c8-5085-44d9-9385-444f547751e4'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:50.147327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-813694c8-5085-44d9-9385-444f547751e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a3318d9534c4527b6d3d4035239e868', 'request_id': 'req-813694c8-5085-44d9-9385-444f547751e4'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:50.178096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-813694c8-5085-44d9-9385-444f547751e4 admin admin] The token cache is being invalidate because project 8a3318d9534c4527b6d3d4035239e868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:50.178566 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-813694c8-5085-44d9-9385-444f547751e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:50.179412 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-813694c8-5085-44d9-9385-444f547751e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:50.181128 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1085/2166] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:49 2022] DELETE /identity/v3/projects/8a3318d9534c4527b6d3d4035239e868 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:50.185698 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813694c8-5085-44d9-9385-444f547751e4 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:50.188920 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-813694c8-5085-44d9-9385-444f547751e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:50.258260 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:50.301340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1082/2167] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:50.304215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125f3ac-8149-4833-bbb4-fd27246b8933 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:50.305740 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-2125f3ac-8149-4833-bbb4-fd27246b8933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:50.327204 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813694c8-5085-44d9-9385-444f547751e4 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:50.332223 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4670e3-003e-4b0f-80b8-56505da3a6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:50.333211 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4670e3-003e-4b0f-80b8-56505da3a6ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:50.333894 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4670e3-003e-4b0f-80b8-56505da3a6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:50.334249 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4670e3-003e-4b0f-80b8-56505da3a6ff admin admin] PATH_INFO: `/v3/users/972582cfce4a4d098b5c0137208c9b7e` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:50.335569 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4670e3-003e-4b0f-80b8-56505da3a6ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=972582cfce4a4d098b5c0137208c9b7e)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:50.337661 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4670e3-003e-4b0f-80b8-56505da3a6ff admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:50.373320 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125f3ac-8149-4833-bbb4-fd27246b8933 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:50.377097 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e08bc1-b9a5-4826-a0dd-2fd4b90362ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:50.377764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e08bc1-b9a5-4826-a0dd-2fd4b90362ff service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:50.378085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e08bc1-b9a5-4826-a0dd-2fd4b90362ff service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:50.378357 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e08bc1-b9a5-4826-a0dd-2fd4b90362ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:50.389264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e08bc1-b9a5-4826-a0dd-2fd4b90362ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:50.391029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e08bc1-b9a5-4826-a0dd-2fd4b90362ff service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:50.400865 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1083/2168] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:40:50.401625 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-9b4670e3-003e-4b0f-80b8-56505da3a6ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '972582cfce4a4d098b5c0137208c9b7e'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:50.411015 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-9b4670e3-003e-4b0f-80b8-56505da3a6ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '972582cfce4a4d098b5c0137208c9b7e'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:50.411871 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-9b4670e3-003e-4b0f-80b8-56505da3a6ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '972582cfce4a4d098b5c0137208c9b7e'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:50.420702 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-9b4670e3-003e-4b0f-80b8-56505da3a6ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '972582cfce4a4d098b5c0137208c9b7e'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:50.421867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-9b4670e3-003e-4b0f-80b8-56505da3a6ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '972582cfce4a4d098b5c0137208c9b7e'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:50.445043 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1086/2169] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:50 2022] DELETE /identity/v3/users/972582cfce4a4d098b5c0137208c9b7e => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:51.185808 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e08bc1-b9a5-4826-a0dd-2fd4b90362ff service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:51.213918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e08bc1-b9a5-4826-a0dd-2fd4b90362ff service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:51.344373 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2e500-35fd-40ed-aee8-9c0ba06e3d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:51.345793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2e500-35fd-40ed-aee8-9c0ba06e3d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:51.346180 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2e500-35fd-40ed-aee8-9c0ba06e3d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:51.346688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2e500-35fd-40ed-aee8-9c0ba06e3d06 admin admin] PATH_INFO: `/v3/projects/ba76ffba63b948fb97f5d040fc41a0ce` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:51.349757 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb2e500-35fd-40ed-aee8-9c0ba06e3d06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba76ffba63b948fb97f5d040fc41a0ce)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:51.352985 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb2e500-35fd-40ed-aee8-9c0ba06e3d06 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:51.505758 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-6bb2e500-35fd-40ed-aee8-9c0ba06e3d06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba76ffba63b948fb97f5d040fc41a0ce', 'request_id': 'req-6bb2e500-35fd-40ed-aee8-9c0ba06e3d06'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:51.518615 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-6bb2e500-35fd-40ed-aee8-9c0ba06e3d06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba76ffba63b948fb97f5d040fc41a0ce', 'request_id': 'req-6bb2e500-35fd-40ed-aee8-9c0ba06e3d06'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:51.533731 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-6bb2e500-35fd-40ed-aee8-9c0ba06e3d06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba76ffba63b948fb97f5d040fc41a0ce', 'request_id': 'req-6bb2e500-35fd-40ed-aee8-9c0ba06e3d06'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:51.550907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-6bb2e500-35fd-40ed-aee8-9c0ba06e3d06 admin admin] The token cache is being invalidate because project ba76ffba63b948fb97f5d040fc41a0ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:51.551456 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-6bb2e500-35fd-40ed-aee8-9c0ba06e3d06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:51.552365 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-6bb2e500-35fd-40ed-aee8-9c0ba06e3d06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:51.560168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1084/2170] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:51 2022] DELETE /identity/v3/projects/ba76ffba63b948fb97f5d040fc41a0ce => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:51.561739 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ec45a-2e29-4ac6-b275-b958b71c65b6 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:51.562675 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ec45a-2e29-4ac6-b275-b958b71c65b6 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:51.562951 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ec45a-2e29-4ac6-b275-b958b71c65b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:51.597634 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-807ec45a-2e29-4ac6-b275-b958b71c65b6 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:51.599052 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-807ec45a-2e29-4ac6-b275-b958b71c65b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:51.644582 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:51.680106 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1087/2171] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:51.688016 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2fd87-4a15-41dc-8505-d1abb0d222c9 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:51.688308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2fd87-4a15-41dc-8505-d1abb0d222c9 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:51.688586 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2fd87-4a15-41dc-8505-d1abb0d222c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:51.708830 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-68c2fd87-4a15-41dc-8505-d1abb0d222c9 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:51.710095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-68c2fd87-4a15-41dc-8505-d1abb0d222c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:51.718434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1085/2172] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:51.723840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ec45a-2e29-4ac6-b275-b958b71c65b6 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:51.736834 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ec45a-2e29-4ac6-b275-b958b71c65b6 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:51.742260 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a75726-3801-447d-8985-04030b317972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:51.742985 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a75726-3801-447d-8985-04030b317972 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:51.743267 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a75726-3801-447d-8985-04030b317972 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:51.743545 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a75726-3801-447d-8985-04030b317972 admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:51.745533 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a75726-3801-447d-8985-04030b317972 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:51.745849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a75726-3801-447d-8985-04030b317972 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:51.747991 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a75726-3801-447d-8985-04030b317972 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:51.763974 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1088/2173] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:51.768643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2fd87-4a15-41dc-8505-d1abb0d222c9 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:51.780794 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2fd87-4a15-41dc-8505-d1abb0d222c9 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:51.785897 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8e014-029a-4e78-8380-2cb12b568b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:51.786584 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8e014-029a-4e78-8380-2cb12b568b3a admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:51.786854 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8e014-029a-4e78-8380-2cb12b568b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:51.787134 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8e014-029a-4e78-8380-2cb12b568b3a admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:51.787843 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc8e014-029a-4e78-8380-2cb12b568b3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:51.789888 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc8e014-029a-4e78-8380-2cb12b568b3a admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:51.807399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:51.831344 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1086/2174] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:51 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:51.832055 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a75726-3801-447d-8985-04030b317972 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:51.845869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a75726-3801-447d-8985-04030b317972 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:51.930576 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8e014-029a-4e78-8380-2cb12b568b3a admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:51.932447 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-2dc8e014-029a-4e78-8380-2cb12b568b3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:51.955457 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41747f86-7730-4b11-b184-fc3d137ed49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:51.956128 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41747f86-7730-4b11-b184-fc3d137ed49f admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:51.956410 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41747f86-7730-4b11-b184-fc3d137ed49f admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:51.956685 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41747f86-7730-4b11-b184-fc3d137ed49f admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:51.957422 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41747f86-7730-4b11-b184-fc3d137ed49f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:51.959467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41747f86-7730-4b11-b184-fc3d137ed49f admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:51.987242 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1089/2175] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:51 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:40:51.987242 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b54f0b-93f2-4bbb-b68b-183cf7d762b4 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:51.987242 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b54f0b-93f2-4bbb-b68b-183cf7d762b4 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:51.987242 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b54f0b-93f2-4bbb-b68b-183cf7d762b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:52.006632 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-88b54f0b-93f2-4bbb-b68b-183cf7d762b4 None None] MFA Rules not processed for user `24306a0e1b3c4ae08576a6b6762af0be`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:52.008081 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-88b54f0b-93f2-4bbb-b68b-183cf7d762b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:52.014509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8e014-029a-4e78-8380-2cb12b568b3a admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:52.019267 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9f931-59e7-408c-8628-594313a0b012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:52.020141 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9f931-59e7-408c-8628-594313a0b012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:52.020449 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9f931-59e7-408c-8628-594313a0b012 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:52.020702 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9f931-59e7-408c-8628-594313a0b012 admin admin] PATH_INFO: `/v3/users/e971b95ceacc43d2ae82da35ec9f7fe5` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:52.021969 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9f931-59e7-408c-8628-594313a0b012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e971b95ceacc43d2ae82da35ec9f7fe5)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:52.023902 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9f931-59e7-408c-8628-594313a0b012 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:52.071543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1090/2176] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:52.072156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b54f0b-93f2-4bbb-b68b-183cf7d762b4 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:52.084697 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b54f0b-93f2-4bbb-b68b-183cf7d762b4 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:52.090219 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6c887-75c4-4c7b-8cd9-4e1d66e06c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:52.090889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6c887-75c4-4c7b-8cd9-4e1d66e06c1e admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:52.091168 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6c887-75c4-4c7b-8cd9-4e1d66e06c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:52.091439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6c887-75c4-4c7b-8cd9-4e1d66e06c1e admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:52.092174 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6c887-75c4-4c7b-8cd9-4e1d66e06c1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:52.093843 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6c887-75c4-4c7b-8cd9-4e1d66e06c1e admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:52.094200 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-03c9f931-59e7-408c-8628-594313a0b012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e971b95ceacc43d2ae82da35ec9f7fe5'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:52.104316 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-03c9f931-59e7-408c-8628-594313a0b012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e971b95ceacc43d2ae82da35ec9f7fe5'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:52.105050 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1091/2177] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:52.105293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-03c9f931-59e7-408c-8628-594313a0b012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e971b95ceacc43d2ae82da35ec9f7fe5'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:52.108828 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6c887-75c4-4c7b-8cd9-4e1d66e06c1e admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:52.110595 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-65f6c887-75c4-4c7b-8cd9-4e1d66e06c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:52.122577 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-03c9f931-59e7-408c-8628-594313a0b012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e971b95ceacc43d2ae82da35ec9f7fe5'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:52.124816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6c887-75c4-4c7b-8cd9-4e1d66e06c1e admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:52.129650 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419306eb-70cf-4efa-803e-63dc5c1bf24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:52.130317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419306eb-70cf-4efa-803e-63dc5c1bf24e admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:52.130596 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419306eb-70cf-4efa-803e-63dc5c1bf24e admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:52.130872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419306eb-70cf-4efa-803e-63dc5c1bf24e admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:52.131604 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419306eb-70cf-4efa-803e-63dc5c1bf24e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:52.133236 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419306eb-70cf-4efa-803e-63dc5c1bf24e admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:52.135481 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-03c9f931-59e7-408c-8628-594313a0b012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e971b95ceacc43d2ae82da35ec9f7fe5'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:52.143106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1087/2178] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:51 2022] DELETE /identity/v3/users/e971b95ceacc43d2ae82da35ec9f7fe5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:52.157226 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1092/2179] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:40:52.163913 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9f931-59e7-408c-8628-594313a0b012 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:52.263375 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9f931-59e7-408c-8628-594313a0b012 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:52.273334 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc119f7-900d-4513-a385-4b9733b9a76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:52.274643 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc119f7-900d-4513-a385-4b9733b9a76d admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:52.274957 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc119f7-900d-4513-a385-4b9733b9a76d admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:52.275685 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc119f7-900d-4513-a385-4b9733b9a76d admin admin] PATH_INFO: `/v3/projects/d9bb1d0f44ed46898a51f3d1be7e5268/users/2d0cffe64ae7414bbc392675fbeed9de/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:52.302615 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc119f7-900d-4513-a385-4b9733b9a76d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9bb1d0f44ed46898a51f3d1be7e5268, user_id=2d0cffe64ae7414bbc392675fbeed9de, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:52.308051 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc119f7-900d-4513-a385-4b9733b9a76d admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:52.312696 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:52.347969 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1088/2180] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:52 2022] PUT /identity/v3/projects/d9bb1d0f44ed46898a51f3d1be7e5268/users/2d0cffe64ae7414bbc392675fbeed9de/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:52.359996 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5788a-b449-4219-8270-0725141adab6 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:52.360245 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5788a-b449-4219-8270-0725141adab6 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:52.360554 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5788a-b449-4219-8270-0725141adab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:52.373210 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc119f7-900d-4513-a385-4b9733b9a76d admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:52.377273 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-3bc119f7-900d-4513-a385-4b9733b9a76d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:52.407232 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-75f5788a-b449-4219-8270-0725141adab6 None None] MFA Rules not processed for user `2d0cffe64ae7414bbc392675fbeed9de`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:52.408650 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-75f5788a-b449-4219-8270-0725141adab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:52.462627 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:52.536440 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1093/2181] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:52.548385 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5788a-b449-4219-8270-0725141adab6 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:52.550261 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-75f5788a-b449-4219-8270-0725141adab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:52.554558 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc119f7-900d-4513-a385-4b9733b9a76d admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:52.563116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65def2-ae5e-4caf-be8a-af76de4451e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:52.564739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65def2-ae5e-4caf-be8a-af76de4451e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:52.565217 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65def2-ae5e-4caf-be8a-af76de4451e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:52.565671 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65def2-ae5e-4caf-be8a-af76de4451e7 admin admin] PATH_INFO: `/v3/projects/f88b4bb8bbb54621ad28c6a75a3d2837` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:52.567600 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f65def2-ae5e-4caf-be8a-af76de4451e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f88b4bb8bbb54621ad28c6a75a3d2837)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:52.569813 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f65def2-ae5e-4caf-be8a-af76de4451e7 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:52.642027 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-3f65def2-ae5e-4caf-be8a-af76de4451e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f88b4bb8bbb54621ad28c6a75a3d2837', 'request_id': 'req-3f65def2-ae5e-4caf-be8a-af76de4451e7'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:52.656427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-3f65def2-ae5e-4caf-be8a-af76de4451e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f88b4bb8bbb54621ad28c6a75a3d2837', 'request_id': 'req-3f65def2-ae5e-4caf-be8a-af76de4451e7'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:52.667305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-3f65def2-ae5e-4caf-be8a-af76de4451e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f88b4bb8bbb54621ad28c6a75a3d2837', 'request_id': 'req-3f65def2-ae5e-4caf-be8a-af76de4451e7'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:52.696530 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-3f65def2-ae5e-4caf-be8a-af76de4451e7 admin admin] The token cache is being invalidate because project f88b4bb8bbb54621ad28c6a75a3d2837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:52.696832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-3f65def2-ae5e-4caf-be8a-af76de4451e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:52.697608 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-3f65def2-ae5e-4caf-be8a-af76de4451e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:52.701009 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1089/2182] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:52 2022] DELETE /identity/v3/projects/f88b4bb8bbb54621ad28c6a75a3d2837 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:52.704816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5788a-b449-4219-8270-0725141adab6 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:52.710933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bc40e-19fb-4266-b48f-50f7ec90867f None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:52.711269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bc40e-19fb-4266-b48f-50f7ec90867f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:52.711553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bc40e-19fb-4266-b48f-50f7ec90867f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:52.737698 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-867bc40e-19fb-4266-b48f-50f7ec90867f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:52.739056 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-867bc40e-19fb-4266-b48f-50f7ec90867f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:52.790436 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638bd64-3a65-427d-99c8-07f6d9484312 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2460e6a7e34a6881984e4f7ed294c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:52.792116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638bd64-3a65-427d-99c8-07f6d9484312 service glance] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:52.792596 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638bd64-3a65-427d-99c8-07f6d9484312 service glance] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:52.793062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638bd64-3a65-427d-99c8-07f6d9484312 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:52.795273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a638bd64-3a65-427d-99c8-07f6d9484312 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:52.819509 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1090/2183] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:52.828149 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0b4ba-37d1-403f-ae4f-d0854f365291 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:52.828372 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0b4ba-37d1-403f-ae4f-d0854f365291 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:52.828629 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0b4ba-37d1-403f-ae4f-d0854f365291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:52.848443 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-4ca0b4ba-37d1-403f-ae4f-d0854f365291 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:52.849221 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-4ca0b4ba-37d1-403f-ae4f-d0854f365291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:52.860197 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1091/2184] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:40:52.866014 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0b4ba-37d1-403f-ae4f-d0854f365291 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:52.874427 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638bd64-3a65-427d-99c8-07f6d9484312 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:52.874623 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0b4ba-37d1-403f-ae4f-d0854f365291 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:52.876002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638bd64-3a65-427d-99c8-07f6d9484312 service glance] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:52.879251 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639430f1-c94a-462f-b6a2-604caec56df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:52.879919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639430f1-c94a-462f-b6a2-604caec56df8 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:52.880210 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639430f1-c94a-462f-b6a2-604caec56df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:52.880475 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639430f1-c94a-462f-b6a2-604caec56df8 admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:52.882049 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639430f1-c94a-462f-b6a2-604caec56df8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:52.882049 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639430f1-c94a-462f-b6a2-604caec56df8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:52.883518 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639430f1-c94a-462f-b6a2-604caec56df8 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:52.887795 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1094/2185] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:52.897753 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1092/2186] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:40:52.902695 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638bd64-3a65-427d-99c8-07f6d9484312 service glance] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:52.923360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638bd64-3a65-427d-99c8-07f6d9484312 service glance] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:52.930631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6af66-56ef-4fd5-9a1d-658bb0b0b84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:52.931950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6af66-56ef-4fd5-9a1d-658bb0b0b84f admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:52.932533 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6af66-56ef-4fd5-9a1d-658bb0b0b84f admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:52.932917 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6af66-56ef-4fd5-9a1d-658bb0b0b84f admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:52.933993 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb6af66-56ef-4fd5-9a1d-658bb0b0b84f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:52.936259 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb6af66-56ef-4fd5-9a1d-658bb0b0b84f admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:52.953616 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:52.976083 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1095/2187] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:52 2022] POST /identity/v3/projects => generated 363 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:52.978381 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639430f1-c94a-462f-b6a2-604caec56df8 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:52.994741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639430f1-c94a-462f-b6a2-604caec56df8 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:53.008622 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:53.071497 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231e588-97db-484a-9e8d-6f071ed7b3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:53.073395 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231e588-97db-484a-9e8d-6f071ed7b3d2 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:53.073786 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231e588-97db-484a-9e8d-6f071ed7b3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:53.074216 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231e588-97db-484a-9e8d-6f071ed7b3d2 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:53.075044 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231e588-97db-484a-9e8d-6f071ed7b3d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:53.078734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231e588-97db-484a-9e8d-6f071ed7b3d2 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:53.109267 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1093/2188] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:52 2022] POST /identity/v3/users => generated 282 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:40:53.110757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6af66-56ef-4fd5-9a1d-658bb0b0b84f admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:53.124022 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6af66-56ef-4fd5-9a1d-658bb0b0b84f admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:53.130881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f34ec-e54e-4d18-8dab-bd151c96dace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:53.131968 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f34ec-e54e-4d18-8dab-bd151c96dace admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:53.132324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f34ec-e54e-4d18-8dab-bd151c96dace admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:53.132718 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f34ec-e54e-4d18-8dab-bd151c96dace admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:53.133619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f34ec-e54e-4d18-8dab-bd151c96dace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:53.136782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f34ec-e54e-4d18-8dab-bd151c96dace admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:53.152508 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1096/2189] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:53.153733 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231e588-97db-484a-9e8d-6f071ed7b3d2 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:53.165899 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231e588-97db-484a-9e8d-6f071ed7b3d2 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:53.172919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981ef15-2097-480d-8cb3-6c06fe13f811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:53.173712 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981ef15-2097-480d-8cb3-6c06fe13f811 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:53.174061 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981ef15-2097-480d-8cb3-6c06fe13f811 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:53.174470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981ef15-2097-480d-8cb3-6c06fe13f811 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:53.175282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981ef15-2097-480d-8cb3-6c06fe13f811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:53.176977 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981ef15-2097-480d-8cb3-6c06fe13f811 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:53.189209 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1094/2190] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:53.193973 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f34ec-e54e-4d18-8dab-bd151c96dace admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:53.206739 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f34ec-e54e-4d18-8dab-bd151c96dace admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:53.211776 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64f6b7-c4b0-4b41-818b-8417bed9ff71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:53.212661 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64f6b7-c4b0-4b41-818b-8417bed9ff71 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:53.213020 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64f6b7-c4b0-4b41-818b-8417bed9ff71 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:53.213330 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64f6b7-c4b0-4b41-818b-8417bed9ff71 admin admin] PATH_INFO: `/v3/projects/ebac0198d5f74999b1ac864de62b13f7/users/d933963e196b4579bfa3abe2d1a5b919/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:53.237239 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64f6b7-c4b0-4b41-818b-8417bed9ff71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebac0198d5f74999b1ac864de62b13f7, user_id=d933963e196b4579bfa3abe2d1a5b919, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:53.240585 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64f6b7-c4b0-4b41-818b-8417bed9ff71 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:53.266852 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1097/2191] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:53 2022] PUT /identity/v3/projects/ebac0198d5f74999b1ac864de62b13f7/users/d933963e196b4579bfa3abe2d1a5b919/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:53.277863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9980ca7-21f3-4407-996c-c1aa6d99257d None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:53.278418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9980ca7-21f3-4407-996c-c1aa6d99257d None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:53.278418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9980ca7-21f3-4407-996c-c1aa6d99257d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:53.311629 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-d9980ca7-21f3-4407-996c-c1aa6d99257d None None] MFA Rules not processed for user `d933963e196b4579bfa3abe2d1a5b919`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:53.313269 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d9980ca7-21f3-4407-996c-c1aa6d99257d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:53.422096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1095/2192] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:53 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:53.427055 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64f6b7-c4b0-4b41-818b-8417bed9ff71 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:53.578753 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64f6b7-c4b0-4b41-818b-8417bed9ff71 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:53.585734 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfe160-bdaf-43a2-8fc7-d86139c12622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:53.586454 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfe160-bdaf-43a2-8fc7-d86139c12622 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:53.586795 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfe160-bdaf-43a2-8fc7-d86139c12622 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:53.587157 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfe160-bdaf-43a2-8fc7-d86139c12622 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:53.588464 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dfe160-bdaf-43a2-8fc7-d86139c12622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:53.590874 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dfe160-bdaf-43a2-8fc7-d86139c12622 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:53.625050 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1098/2193] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:53 2022] POST /identity/v3/projects => generated 365 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:53.630458 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9980ca7-21f3-4407-996c-c1aa6d99257d None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:53.645605 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9980ca7-21f3-4407-996c-c1aa6d99257d None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:53.722321 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99172c2-5758-4cdc-b9d0-e9cf2859d750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:53.723087 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99172c2-5758-4cdc-b9d0-e9cf2859d750 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:53.723439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99172c2-5758-4cdc-b9d0-e9cf2859d750 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:53.723788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99172c2-5758-4cdc-b9d0-e9cf2859d750 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:53.724550 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99172c2-5758-4cdc-b9d0-e9cf2859d750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:53.726561 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99172c2-5758-4cdc-b9d0-e9cf2859d750 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:53.752491 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1096/2194] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:53 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:53.753849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfe160-bdaf-43a2-8fc7-d86139c12622 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:53.765434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfe160-bdaf-43a2-8fc7-d86139c12622 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:53.779218 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93affd-64b6-4fc5-b287-5831d67512ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:53.780427 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93affd-64b6-4fc5-b287-5831d67512ce admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:53.780695 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93affd-64b6-4fc5-b287-5831d67512ce admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:53.781391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93affd-64b6-4fc5-b287-5831d67512ce admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:53.782694 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93affd-64b6-4fc5-b287-5831d67512ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:53.785210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93affd-64b6-4fc5-b287-5831d67512ce admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:53.803300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1099/2195] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:40:53.809609 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99172c2-5758-4cdc-b9d0-e9cf2859d750 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:53.819681 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:53.826582 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99172c2-5758-4cdc-b9d0-e9cf2859d750 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:53.843427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb477f-fe2e-40d7-bfab-5e3ac29e2b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:53.845200 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb477f-fe2e-40d7-bfab-5e3ac29e2b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:53.845924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb477f-fe2e-40d7-bfab-5e3ac29e2b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:53.845924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb477f-fe2e-40d7-bfab-5e3ac29e2b2a admin admin] PATH_INFO: `/v3/projects/161fdfe0f78b4aafba1b0d16df8b4c64/users/675ef75ee32c4bb48263fa07524875a1/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:53.908082 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb477f-fe2e-40d7-bfab-5e3ac29e2b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=161fdfe0f78b4aafba1b0d16df8b4c64, user_id=675ef75ee32c4bb48263fa07524875a1, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:53.916896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb477f-fe2e-40d7-bfab-5e3ac29e2b2a admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:53.947018 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1097/2196] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:53 2022] PUT /identity/v3/projects/161fdfe0f78b4aafba1b0d16df8b4c64/users/675ef75ee32c4bb48263fa07524875a1/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:53.951508 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93affd-64b6-4fc5-b287-5831d67512ce admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:53.967155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93affd-64b6-4fc5-b287-5831d67512ce admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:54.095382 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241d18a-6874-4538-88be-10f846689383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:54.095978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241d18a-6874-4538-88be-10f846689383 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:54.096257 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241d18a-6874-4538-88be-10f846689383 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:54.096529 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241d18a-6874-4538-88be-10f846689383 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:54.097328 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241d18a-6874-4538-88be-10f846689383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:54.098952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241d18a-6874-4538-88be-10f846689383 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:54.110361 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1100/2197] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:54.114851 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb477f-fe2e-40d7-bfab-5e3ac29e2b2a admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:54.126251 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb477f-fe2e-40d7-bfab-5e3ac29e2b2a admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:54.132332 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df6645-ece0-4ae3-8722-89ae7a6324ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:54.133451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df6645-ece0-4ae3-8722-89ae7a6324ec admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:54.133737 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df6645-ece0-4ae3-8722-89ae7a6324ec admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:54.134012 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df6645-ece0-4ae3-8722-89ae7a6324ec admin admin] PATH_INFO: `/v3/projects/161fdfe0f78b4aafba1b0d16df8b4c64/users/675ef75ee32c4bb48263fa07524875a1/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:54.136298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df6645-ece0-4ae3-8722-89ae7a6324ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=161fdfe0f78b4aafba1b0d16df8b4c64, user_id=675ef75ee32c4bb48263fa07524875a1, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:54.139033 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df6645-ece0-4ae3-8722-89ae7a6324ec admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:54.161828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1098/2198] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:54 2022] PUT /identity/v3/projects/161fdfe0f78b4aafba1b0d16df8b4c64/users/675ef75ee32c4bb48263fa07524875a1/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:54.174194 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615ca68-f991-45bb-8b36-9e02abf1f40b None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:54.174492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615ca68-f991-45bb-8b36-9e02abf1f40b None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:54.174777 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615ca68-f991-45bb-8b36-9e02abf1f40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:54.204581 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-9615ca68-f991-45bb-8b36-9e02abf1f40b None None] MFA Rules not processed for user `675ef75ee32c4bb48263fa07524875a1`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:54.205927 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-9615ca68-f991-45bb-8b36-9e02abf1f40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:54.234299 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:54.331953 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1101/2199] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:54 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:54.347632 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df6645-ece0-4ae3-8722-89ae7a6324ec admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:54.351238 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-29df6645-ece0-4ae3-8722-89ae7a6324ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:54.449690 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df6645-ece0-4ae3-8722-89ae7a6324ec admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:54.460356 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d46b55-9251-4711-8918-7f9436751abc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:54.461763 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d46b55-9251-4711-8918-7f9436751abc service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:54.462048 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d46b55-9251-4711-8918-7f9436751abc service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:54.462825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d46b55-9251-4711-8918-7f9436751abc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:54.482844 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d46b55-9251-4711-8918-7f9436751abc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:54.484548 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d46b55-9251-4711-8918-7f9436751abc service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:54.497388 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1099/2200] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:54 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:54.584532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615ca68-f991-45bb-8b36-9e02abf1f40b None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:54.586089 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-9615ca68-f991-45bb-8b36-9e02abf1f40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:54.599631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615ca68-f991-45bb-8b36-9e02abf1f40b None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:54.603345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feac9f6-aa8e-4840-8db3-6de8ef5e7d78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:54.604196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feac9f6-aa8e-4840-8db3-6de8ef5e7d78 service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:54.604509 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feac9f6-aa8e-4840-8db3-6de8ef5e7d78 service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:54.604811 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feac9f6-aa8e-4840-8db3-6de8ef5e7d78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:54.667595 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feac9f6-aa8e-4840-8db3-6de8ef5e7d78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:54.669115 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feac9f6-aa8e-4840-8db3-6de8ef5e7d78 service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:54.682610 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1102/2201] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:54 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:55.235370 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d46b55-9251-4711-8918-7f9436751abc service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:55.236902 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-36d46b55-9251-4711-8918-7f9436751abc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:55.372291 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d46b55-9251-4711-8918-7f9436751abc service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:55.383778 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ec4a3-5ea4-43df-9849-15774cbb80ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:55.385215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ec4a3-5ea4-43df-9849-15774cbb80ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:55.385551 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ec4a3-5ea4-43df-9849-15774cbb80ab admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:55.385917 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ec4a3-5ea4-43df-9849-15774cbb80ab admin admin] PATH_INFO: `/v3/users/57cc4cf540d54483b498c47f9a55efcb` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:55.387258 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ec4a3-5ea4-43df-9849-15774cbb80ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=57cc4cf540d54483b498c47f9a55efcb)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:55.390423 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ec4a3-5ea4-43df-9849-15774cbb80ab admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:55.498283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-201ec4a3-5ea4-43df-9849-15774cbb80ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57cc4cf540d54483b498c47f9a55efcb'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:55.510748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-201ec4a3-5ea4-43df-9849-15774cbb80ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57cc4cf540d54483b498c47f9a55efcb'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:55.512459 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-201ec4a3-5ea4-43df-9849-15774cbb80ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57cc4cf540d54483b498c47f9a55efcb'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:55.536973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-201ec4a3-5ea4-43df-9849-15774cbb80ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57cc4cf540d54483b498c47f9a55efcb'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:55.539365 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:55.555404 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-201ec4a3-5ea4-43df-9849-15774cbb80ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57cc4cf540d54483b498c47f9a55efcb'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:55.578406 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1100/2202] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:55 2022] DELETE /identity/v3/users/57cc4cf540d54483b498c47f9a55efcb => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:55.583056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04016aaa-523a-477a-835b-be603859f3d6 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:55.583978 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04016aaa-523a-477a-835b-be603859f3d6 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:55.584445 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04016aaa-523a-477a-835b-be603859f3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:55.630698 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-04016aaa-523a-477a-835b-be603859f3d6 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:55.632289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-04016aaa-523a-477a-835b-be603859f3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:55.701264 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1103/2203] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:55.710976 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ec4a3-5ea4-43df-9849-15774cbb80ab admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:55.712544 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-201ec4a3-5ea4-43df-9849-15774cbb80ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:55.783496 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ec4a3-5ea4-43df-9849-15774cbb80ab admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:55.787548 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc43bb4-9acc-436b-a423-ad28a6c2adca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:55.788526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc43bb4-9acc-436b-a423-ad28a6c2adca service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:55.788886 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc43bb4-9acc-436b-a423-ad28a6c2adca service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:55.789279 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc43bb4-9acc-436b-a423-ad28a6c2adca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:55.800738 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc43bb4-9acc-436b-a423-ad28a6c2adca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:55.802369 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc43bb4-9acc-436b-a423-ad28a6c2adca service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:55.816094 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1101/2204] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:56.222036 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04016aaa-523a-477a-835b-be603859f3d6 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:56.223815 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-04016aaa-523a-477a-835b-be603859f3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:56.258603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04016aaa-523a-477a-835b-be603859f3d6 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:56.267915 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2b885-9755-444c-9ced-ff91239dd281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:56.269102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2b885-9755-444c-9ced-ff91239dd281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:56.269522 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2b885-9755-444c-9ced-ff91239dd281 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:56.269762 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2b885-9755-444c-9ced-ff91239dd281 admin admin] PATH_INFO: `/v3/projects/c84341691bfd4342a1460e6c27a34df6` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:56.271082 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2b885-9755-444c-9ced-ff91239dd281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c84341691bfd4342a1460e6c27a34df6)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:56.273178 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2b885-9755-444c-9ced-ff91239dd281 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:56.341786 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-dcc2b885-9755-444c-9ced-ff91239dd281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c84341691bfd4342a1460e6c27a34df6', 'request_id': 'req-dcc2b885-9755-444c-9ced-ff91239dd281'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:56.345207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:56.355356 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-dcc2b885-9755-444c-9ced-ff91239dd281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c84341691bfd4342a1460e6c27a34df6', 'request_id': 'req-dcc2b885-9755-444c-9ced-ff91239dd281'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:56.364360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-dcc2b885-9755-444c-9ced-ff91239dd281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c84341691bfd4342a1460e6c27a34df6', 'request_id': 'req-dcc2b885-9755-444c-9ced-ff91239dd281'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:56.379469 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-dcc2b885-9755-444c-9ced-ff91239dd281 admin admin] The token cache is being invalidate because project c84341691bfd4342a1460e6c27a34df6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:56.380494 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-dcc2b885-9755-444c-9ced-ff91239dd281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:56.381983 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-dcc2b885-9755-444c-9ced-ff91239dd281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:56.388363 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1104/2205] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:56 2022] DELETE /identity/v3/projects/c84341691bfd4342a1460e6c27a34df6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:56.390160 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827db1a-2354-44e7-a26b-204a6fb199b4 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:56.390635 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827db1a-2354-44e7-a26b-204a6fb199b4 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:56.390891 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827db1a-2354-44e7-a26b-204a6fb199b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:56.413039 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-2827db1a-2354-44e7-a26b-204a6fb199b4 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:56.414401 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-2827db1a-2354-44e7-a26b-204a6fb199b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:56.511173 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1102/2206] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:56.519310 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb392e-45ce-4808-abd7-5d7b8da51a1f None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:56.519944 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb392e-45ce-4808-abd7-5d7b8da51a1f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:56.520228 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb392e-45ce-4808-abd7-5d7b8da51a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:56.543790 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-3feb392e-45ce-4808-abd7-5d7b8da51a1f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:56.545053 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3feb392e-45ce-4808-abd7-5d7b8da51a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:56.554579 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1105/2207] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:56.560194 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827db1a-2354-44e7-a26b-204a6fb199b4 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:56.570716 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827db1a-2354-44e7-a26b-204a6fb199b4 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:56.576138 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22038bd5-81ca-4722-ab9f-cafb870167ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:56.577713 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22038bd5-81ca-4722-ab9f-cafb870167ca admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:56.577994 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22038bd5-81ca-4722-ab9f-cafb870167ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:56.578270 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22038bd5-81ca-4722-ab9f-cafb870167ca admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:56.579656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22038bd5-81ca-4722-ab9f-cafb870167ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:56.579971 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22038bd5-81ca-4722-ab9f-cafb870167ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:56.581764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22038bd5-81ca-4722-ab9f-cafb870167ca admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:56.599764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1103/2208] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:56.601478 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb392e-45ce-4808-abd7-5d7b8da51a1f None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:56.611827 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb392e-45ce-4808-abd7-5d7b8da51a1f None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:56.617285 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34572fce-57da-4285-9123-bb55712b2d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:56.617999 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34572fce-57da-4285-9123-bb55712b2d13 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:56.618286 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34572fce-57da-4285-9123-bb55712b2d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:56.618559 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34572fce-57da-4285-9123-bb55712b2d13 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:56.619805 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34572fce-57da-4285-9123-bb55712b2d13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:56.621816 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34572fce-57da-4285-9123-bb55712b2d13 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:56.654591 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1106/2209] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:56 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:56.662189 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22038bd5-81ca-4722-ab9f-cafb870167ca admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:56.678693 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22038bd5-81ca-4722-ab9f-cafb870167ca admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:56.771714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422df3bf-cb11-4a52-af03-cfbfb39526ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:56.773455 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422df3bf-cb11-4a52-af03-cfbfb39526ac admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:56.773741 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422df3bf-cb11-4a52-af03-cfbfb39526ac admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:56.774018 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422df3bf-cb11-4a52-af03-cfbfb39526ac admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:56.775229 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422df3bf-cb11-4a52-af03-cfbfb39526ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:56.776674 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422df3bf-cb11-4a52-af03-cfbfb39526ac admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:56.806135 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1104/2210] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:56 2022] POST /identity/v3/users => generated 288 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:56.806517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34572fce-57da-4285-9123-bb55712b2d13 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:56.818408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34572fce-57da-4285-9123-bb55712b2d13 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:56.827983 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879acb0f-3138-464d-ad6b-952d88749eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:56.828941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879acb0f-3138-464d-ad6b-952d88749eae admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:56.829411 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879acb0f-3138-464d-ad6b-952d88749eae admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:56.829712 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879acb0f-3138-464d-ad6b-952d88749eae admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:56.830465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879acb0f-3138-464d-ad6b-952d88749eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:56.832162 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879acb0f-3138-464d-ad6b-952d88749eae admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:56.852027 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1107/2211] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:56.853177 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422df3bf-cb11-4a52-af03-cfbfb39526ac admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:56.872788 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422df3bf-cb11-4a52-af03-cfbfb39526ac admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:56.879965 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da089a-5350-49d5-bc9d-39c33c1e985a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:56.882301 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da089a-5350-49d5-bc9d-39c33c1e985a admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:56.882705 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da089a-5350-49d5-bc9d-39c33c1e985a admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:56.883092 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da089a-5350-49d5-bc9d-39c33c1e985a admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:56.883957 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da089a-5350-49d5-bc9d-39c33c1e985a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:56.885785 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da089a-5350-49d5-bc9d-39c33c1e985a admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:56.892309 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:56.910574 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879acb0f-3138-464d-ad6b-952d88749eae admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:56.915143 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1105/2212] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:56.932446 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879acb0f-3138-464d-ad6b-952d88749eae admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:56.937841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7b191-d27d-49df-aa7e-d53ff0f182ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:56.938744 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7b191-d27d-49df-aa7e-d53ff0f182ca admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:56.939025 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7b191-d27d-49df-aa7e-d53ff0f182ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:56.939477 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7b191-d27d-49df-aa7e-d53ff0f182ca admin admin] PATH_INFO: `/v3/projects/1601306907ce4c3a8ac7abe15538b2e7/users/e9df616033f146379cc0c9fd8392e459/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:56.955876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7b191-d27d-49df-aa7e-d53ff0f182ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=1601306907ce4c3a8ac7abe15538b2e7, user_id=e9df616033f146379cc0c9fd8392e459, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:56.958702 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7b191-d27d-49df-aa7e-d53ff0f182ca admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:56.975786 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1108/2213] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:40:56 2022] PUT /identity/v3/projects/1601306907ce4c3a8ac7abe15538b2e7/users/e9df616033f146379cc0c9fd8392e459/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:56.987767 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7f9e0-0356-47b4-b36d-200431c4cee3 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:56.988057 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7f9e0-0356-47b4-b36d-200431c4cee3 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:56.988342 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7f9e0-0356-47b4-b36d-200431c4cee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:57.015754 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-60f7f9e0-0356-47b4-b36d-200431c4cee3 None None] MFA Rules not processed for user `e9df616033f146379cc0c9fd8392e459`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:57.017035 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-60f7f9e0-0356-47b4-b36d-200431c4cee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:57.091441 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1106/2214] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:57.102952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7b191-d27d-49df-aa7e-d53ff0f182ca admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:57.105196 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-5cf7b191-d27d-49df-aa7e-d53ff0f182ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:57.181505 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7b191-d27d-49df-aa7e-d53ff0f182ca admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:57.185648 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc330205-54cb-4ba9-9cac-65a22effceef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:57.186314 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc330205-54cb-4ba9-9cac-65a22effceef service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:57.186602 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc330205-54cb-4ba9-9cac-65a22effceef service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:57.186877 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc330205-54cb-4ba9-9cac-65a22effceef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:57.196554 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc330205-54cb-4ba9-9cac-65a22effceef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:57.198180 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc330205-54cb-4ba9-9cac-65a22effceef service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:57.208068 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1109/2215] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:40:58.334486 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81643edd-67a1-496d-9edd-db123090532f None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:58.335015 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81643edd-67a1-496d-9edd-db123090532f None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:58.335129 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81643edd-67a1-496d-9edd-db123090532f None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:58.336515 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1107/2216] 10.222.0.71 () {54 vars in 921 bytes} [Thu Jul 21 20:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:58.360277 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5491de6-31a9-4cb3-a590-e5bba845afbf None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:58.360805 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5491de6-31a9-4cb3-a590-e5bba845afbf None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:58.360915 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5491de6-31a9-4cb3-a590-e5bba845afbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:58.390941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-b5491de6-31a9-4cb3-a590-e5bba845afbf None None] MFA Rules not processed for user `364e80ec79b94812a1df76c62c2b13a0`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:58.392174 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-b5491de6-31a9-4cb3-a590-e5bba845afbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:58.459869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81643edd-67a1-496d-9edd-db123090532f None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:58.471624 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-81643edd-67a1-496d-9edd-db123090532f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:58.482383 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1110/2217] 10.222.0.71 () {58 vars in 1018 bytes} [Thu Jul 21 20:40:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:58.494887 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5491de6-31a9-4cb3-a590-e5bba845afbf None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:58.496542 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-b5491de6-31a9-4cb3-a590-e5bba845afbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:58.510225 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5491de6-31a9-4cb3-a590-e5bba845afbf None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:58.515525 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895b627-bdf1-4d83-82b7-421e6f576c6c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '364e80ec79b94812a1df76c62c2b13a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:58.516270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895b627-bdf1-4d83-82b7-421e6f576c6c service placement] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:58.516633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895b627-bdf1-4d83-82b7-421e6f576c6c service placement] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:58.516981 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895b627-bdf1-4d83-82b7-421e6f576c6c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:58.526453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895b627-bdf1-4d83-82b7-421e6f576c6c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:58.528059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895b627-bdf1-4d83-82b7-421e6f576c6c service placement] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:58.533937 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:58.541520 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1111/2218] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:40:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:40:58.550931 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81643edd-67a1-496d-9edd-db123090532f None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:58.555815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b022a-00d1-447d-ac02-931a13cff793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:58.556978 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b022a-00d1-447d-ac02-931a13cff793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:58.557298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b022a-00d1-447d-ac02-931a13cff793 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:58.557574 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b022a-00d1-447d-ac02-931a13cff793 admin admin] PATH_INFO: `/v3/users/f8fd0fab9dc3448eb7853d809a6dcf63` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:58.558958 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b022a-00d1-447d-ac02-931a13cff793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8fd0fab9dc3448eb7853d809a6dcf63)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:58.560963 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b022a-00d1-447d-ac02-931a13cff793 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:58.614784 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-c51b022a-00d1-447d-ac02-931a13cff793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8fd0fab9dc3448eb7853d809a6dcf63'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:58.625383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-c51b022a-00d1-447d-ac02-931a13cff793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8fd0fab9dc3448eb7853d809a6dcf63'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:58.625383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-c51b022a-00d1-447d-ac02-931a13cff793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8fd0fab9dc3448eb7853d809a6dcf63'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:58.639157 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-c51b022a-00d1-447d-ac02-931a13cff793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8fd0fab9dc3448eb7853d809a6dcf63'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:58.646270 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-c51b022a-00d1-447d-ac02-931a13cff793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8fd0fab9dc3448eb7853d809a6dcf63'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:58.653261 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1108/2219] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:58 2022] DELETE /identity/v3/users/f8fd0fab9dc3448eb7853d809a6dcf63 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:58.849056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895b627-bdf1-4d83-82b7-421e6f576c6c service placement] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:58.877247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895b627-bdf1-4d83-82b7-421e6f576c6c service placement] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:58.952776 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360a097-7747-4551-944b-88ff37232b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:58.954373 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360a097-7747-4551-944b-88ff37232b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:58.954845 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360a097-7747-4551-944b-88ff37232b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:58.955300 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360a097-7747-4551-944b-88ff37232b3f admin admin] PATH_INFO: `/v3/projects/38bdb820cf8c45379b6dced69e7cbfac` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:58.956889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360a097-7747-4551-944b-88ff37232b3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=38bdb820cf8c45379b6dced69e7cbfac)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:58.959016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360a097-7747-4551-944b-88ff37232b3f admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:59.014490 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-f360a097-7747-4551-944b-88ff37232b3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38bdb820cf8c45379b6dced69e7cbfac', 'request_id': 'req-f360a097-7747-4551-944b-88ff37232b3f'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:59.023050 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-f360a097-7747-4551-944b-88ff37232b3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38bdb820cf8c45379b6dced69e7cbfac', 'request_id': 'req-f360a097-7747-4551-944b-88ff37232b3f'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:59.029143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-f360a097-7747-4551-944b-88ff37232b3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38bdb820cf8c45379b6dced69e7cbfac', 'request_id': 'req-f360a097-7747-4551-944b-88ff37232b3f'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:59.042008 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-f360a097-7747-4551-944b-88ff37232b3f admin admin] The token cache is being invalidate because project 38bdb820cf8c45379b6dced69e7cbfac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:59.042299 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-f360a097-7747-4551-944b-88ff37232b3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:59.042975 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-f360a097-7747-4551-944b-88ff37232b3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:59.044493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1112/2220] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:58 2022] DELETE /identity/v3/projects/38bdb820cf8c45379b6dced69e7cbfac => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:59.049290 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b022a-00d1-447d-ac02-931a13cff793 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:59.051116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c51b022a-00d1-447d-ac02-931a13cff793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:59.066040 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b022a-00d1-447d-ac02-931a13cff793 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:59.070071 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:59.132351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2ab4d-3237-4bba-af73-c8468dcca718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:59.133331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2ab4d-3237-4bba-af73-c8468dcca718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:59.133617 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2ab4d-3237-4bba-af73-c8468dcca718 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:59.133898 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2ab4d-3237-4bba-af73-c8468dcca718 admin admin] PATH_INFO: `/v3/users/883d74fad6934dc8a11200452c69b4cd` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:59.135302 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2ab4d-3237-4bba-af73-c8468dcca718 admin admin] RBAC: Authorizing `identity:delete_user(user_id=883d74fad6934dc8a11200452c69b4cd)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:59.137430 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2ab4d-3237-4bba-af73-c8468dcca718 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:59.189952 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-04c2ab4d-3237-4bba-af73-c8468dcca718 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '883d74fad6934dc8a11200452c69b4cd'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:59.197581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-04c2ab4d-3237-4bba-af73-c8468dcca718 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '883d74fad6934dc8a11200452c69b4cd'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:59.198211 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-04c2ab4d-3237-4bba-af73-c8468dcca718 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '883d74fad6934dc8a11200452c69b4cd'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:59.215858 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-04c2ab4d-3237-4bba-af73-c8468dcca718 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '883d74fad6934dc8a11200452c69b4cd'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:59.225371 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-04c2ab4d-3237-4bba-af73-c8468dcca718 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '883d74fad6934dc8a11200452c69b4cd'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:59.232499 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1109/2221] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:40:59 2022] DELETE /identity/v3/users/883d74fad6934dc8a11200452c69b4cd => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:59.429647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360a097-7747-4551-944b-88ff37232b3f admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:59.432270 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f360a097-7747-4551-944b-88ff37232b3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:59.454801 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360a097-7747-4551-944b-88ff37232b3f admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:59.527861 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58660c-d7c1-445c-ba00-10e992a17e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:59.528831 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58660c-d7c1-445c-ba00-10e992a17e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:59.529229 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58660c-d7c1-445c-ba00-10e992a17e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:59.529584 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58660c-d7c1-445c-ba00-10e992a17e8a admin admin] PATH_INFO: `/v3/projects/8b29cb1663a640dcbcfdd8cfcbb22982` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:59.530897 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58660c-d7c1-445c-ba00-10e992a17e8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b29cb1663a640dcbcfdd8cfcbb22982)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:59.532910 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58660c-d7c1-445c-ba00-10e992a17e8a admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:59.577780 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:59.592068 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-4f58660c-d7c1-445c-ba00-10e992a17e8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b29cb1663a640dcbcfdd8cfcbb22982', 'request_id': 'req-4f58660c-d7c1-445c-ba00-10e992a17e8a'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:59.600007 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-4f58660c-d7c1-445c-ba00-10e992a17e8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b29cb1663a640dcbcfdd8cfcbb22982', 'request_id': 'req-4f58660c-d7c1-445c-ba00-10e992a17e8a'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:59.605982 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-4f58660c-d7c1-445c-ba00-10e992a17e8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b29cb1663a640dcbcfdd8cfcbb22982', 'request_id': 'req-4f58660c-d7c1-445c-ba00-10e992a17e8a'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:59.617512 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-4f58660c-d7c1-445c-ba00-10e992a17e8a admin admin] The token cache is being invalidate because project 8b29cb1663a640dcbcfdd8cfcbb22982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:40:59.617809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-4f58660c-d7c1-445c-ba00-10e992a17e8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:59.618469 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-4f58660c-d7c1-445c-ba00-10e992a17e8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:40:59.620905 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1113/2222] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:40:59 2022] DELETE /identity/v3/projects/8b29cb1663a640dcbcfdd8cfcbb22982 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:59.625644 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1278b-262d-46f4-87d7-04e6d53c2ef8 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:59.626146 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1278b-262d-46f4-87d7-04e6d53c2ef8 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:59.626340 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1278b-262d-46f4-87d7-04e6d53c2ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:59.648462 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-ccd1278b-262d-46f4-87d7-04e6d53c2ef8 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:59.649749 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-ccd1278b-262d-46f4-87d7-04e6d53c2ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:59.711686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1110/2223] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:59.719664 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2a036-9268-437b-a022-6c571ecdd0a9 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:59.720209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2a036-9268-437b-a022-6c571ecdd0a9 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:59.720516 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2a036-9268-437b-a022-6c571ecdd0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:59.739785 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-d2c2a036-9268-437b-a022-6c571ecdd0a9 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:59.740993 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-d2c2a036-9268-437b-a022-6c571ecdd0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:59.749553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1114/2224] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:40:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:40:59.753864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1278b-262d-46f4-87d7-04e6d53c2ef8 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:59.764264 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1278b-262d-46f4-87d7-04e6d53c2ef8 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:59.768867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f6023-5bad-4a0a-b4e4-d7bfe804247f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:59.769571 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f6023-5bad-4a0a-b4e4-d7bfe804247f admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:59.769853 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f6023-5bad-4a0a-b4e4-d7bfe804247f admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:59.770128 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f6023-5bad-4a0a-b4e4-d7bfe804247f admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:59.770852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f6023-5bad-4a0a-b4e4-d7bfe804247f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:59.771159 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f6023-5bad-4a0a-b4e4-d7bfe804247f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:59.772759 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f6023-5bad-4a0a-b4e4-d7bfe804247f admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:59.784951 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1111/2225] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:59.789506 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2a036-9268-437b-a022-6c571ecdd0a9 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:59.799302 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2a036-9268-437b-a022-6c571ecdd0a9 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:59.804038 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af339a-03ec-4a84-98dc-968e46daa45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:59.804682 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af339a-03ec-4a84-98dc-968e46daa45c admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:59.804995 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af339a-03ec-4a84-98dc-968e46daa45c admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:59.805297 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af339a-03ec-4a84-98dc-968e46daa45c admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:59.805987 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af339a-03ec-4a84-98dc-968e46daa45c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:59.808505 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af339a-03ec-4a84-98dc-968e46daa45c admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:59.833279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1115/2226] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:40:59 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:59.837917 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f6023-5bad-4a0a-b4e4-d7bfe804247f admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:59.848285 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f6023-5bad-4a0a-b4e4-d7bfe804247f admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:59.857608 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:59.907358 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c3d71-d479-4941-bf2f-d5b1a7a00393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:59.908024 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c3d71-d479-4941-bf2f-d5b1a7a00393 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:59.908304 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c3d71-d479-4941-bf2f-d5b1a7a00393 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:59.908586 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c3d71-d479-4941-bf2f-d5b1a7a00393 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:59.909307 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c3d71-d479-4941-bf2f-d5b1a7a00393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:59.911206 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c3d71-d479-4941-bf2f-d5b1a7a00393 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:59.931545 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1112/2227] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:40:59 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:40:59.936177 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af339a-03ec-4a84-98dc-968e46daa45c admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:59.946492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af339a-03ec-4a84-98dc-968e46daa45c admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:59.951155 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4af03-438d-4aa1-8697-2ce18b2f6833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:59.951812 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4af03-438d-4aa1-8697-2ce18b2f6833 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:59.952086 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4af03-438d-4aa1-8697-2ce18b2f6833 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:59.952359 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4af03-438d-4aa1-8697-2ce18b2f6833 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:59.953109 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4af03-438d-4aa1-8697-2ce18b2f6833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:59.954719 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4af03-438d-4aa1-8697-2ce18b2f6833 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:59.966513 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1116/2228] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:40:59.970822 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c3d71-d479-4941-bf2f-d5b1a7a00393 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:59.981517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c3d71-d479-4941-bf2f-d5b1a7a00393 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:59.986169 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4b1de-6334-418c-85c4-23a83ff8c524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:59.986837 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4b1de-6334-418c-85c4-23a83ff8c524 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:59.987122 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4b1de-6334-418c-85c4-23a83ff8c524 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:59.987402 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4b1de-6334-418c-85c4-23a83ff8c524 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:59.988130 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4b1de-6334-418c-85c4-23a83ff8c524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:59.989760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4b1de-6334-418c-85c4-23a83ff8c524 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:00.000606 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1113/2229] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:41:00.005102 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4af03-438d-4aa1-8697-2ce18b2f6833 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:00.015141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4af03-438d-4aa1-8697-2ce18b2f6833 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:00.019747 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad8eb8-e42d-4e5c-b564-83138d7bec17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:00.020615 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad8eb8-e42d-4e5c-b564-83138d7bec17 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:00.020895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad8eb8-e42d-4e5c-b564-83138d7bec17 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:00.021206 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad8eb8-e42d-4e5c-b564-83138d7bec17 admin admin] PATH_INFO: `/v3/projects/7ed6e7435e2f4e2484eca25601ced86c/users/b63c94bd9e7a488fa9d250a0eb03e185/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:00.037233 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad8eb8-e42d-4e5c-b564-83138d7bec17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ed6e7435e2f4e2484eca25601ced86c, user_id=b63c94bd9e7a488fa9d250a0eb03e185, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:00.040386 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad8eb8-e42d-4e5c-b564-83138d7bec17 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:00.054703 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1117/2230] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:41:00 2022] PUT /identity/v3/projects/7ed6e7435e2f4e2484eca25601ced86c/users/b63c94bd9e7a488fa9d250a0eb03e185/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:41:00.066614 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dc2c0-2009-409e-aa66-f5e803ec0dc2 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:00.066895 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dc2c0-2009-409e-aa66-f5e803ec0dc2 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:00.067222 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dc2c0-2009-409e-aa66-f5e803ec0dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:00.098427 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-396dc2c0-2009-409e-aa66-f5e803ec0dc2 None None] MFA Rules not processed for user `b63c94bd9e7a488fa9d250a0eb03e185`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:00.099677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-396dc2c0-2009-409e-aa66-f5e803ec0dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:00.164814 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1114/2231] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:00.178603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad8eb8-e42d-4e5c-b564-83138d7bec17 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:00.180306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-e1ad8eb8-e42d-4e5c-b564-83138d7bec17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:00.237397 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad8eb8-e42d-4e5c-b564-83138d7bec17 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:00.241008 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62918383-96e8-4d0d-ae34-e58d0d4f2a28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:00.241677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62918383-96e8-4d0d-ae34-e58d0d4f2a28 service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:00.241958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62918383-96e8-4d0d-ae34-e58d0d4f2a28 service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:00.242231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62918383-96e8-4d0d-ae34-e58d0d4f2a28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:00.250888 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62918383-96e8-4d0d-ae34-e58d0d4f2a28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:00.252377 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62918383-96e8-4d0d-ae34-e58d0d4f2a28 service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:00.261702 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1118/2232] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:41:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:41:02.446954 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396dc2c0-2009-409e-aa66-f5e803ec0dc2 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:02.449004 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-396dc2c0-2009-409e-aa66-f5e803ec0dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:02.531847 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396dc2c0-2009-409e-aa66-f5e803ec0dc2 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:02.536500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42f428-33d3-4312-b41b-fa1c4e3e6c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:02.538640 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42f428-33d3-4312-b41b-fa1c4e3e6c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:02.538640 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42f428-33d3-4312-b41b-fa1c4e3e6c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:02.538640 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42f428-33d3-4312-b41b-fa1c4e3e6c7c admin admin] PATH_INFO: `/v3/users/e9df616033f146379cc0c9fd8392e459` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:02.539293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42f428-33d3-4312-b41b-fa1c4e3e6c7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9df616033f146379cc0c9fd8392e459)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:02.541905 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42f428-33d3-4312-b41b-fa1c4e3e6c7c admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:02.567177 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:02.596588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-eb42f428-33d3-4312-b41b-fa1c4e3e6c7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9df616033f146379cc0c9fd8392e459'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:02.603908 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-eb42f428-33d3-4312-b41b-fa1c4e3e6c7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9df616033f146379cc0c9fd8392e459'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:02.604580 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-eb42f428-33d3-4312-b41b-fa1c4e3e6c7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9df616033f146379cc0c9fd8392e459'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:02.623242 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-eb42f428-33d3-4312-b41b-fa1c4e3e6c7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9df616033f146379cc0c9fd8392e459'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:02.630071 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-eb42f428-33d3-4312-b41b-fa1c4e3e6c7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9df616033f146379cc0c9fd8392e459'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:02.637032 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1115/2233] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:41:02 2022] DELETE /identity/v3/users/e9df616033f146379cc0c9fd8392e459 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:41:02.643044 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43860ca0-be67-41fc-a663-aa553b799c69 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:02.643630 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43860ca0-be67-41fc-a663-aa553b799c69 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:02.643630 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43860ca0-be67-41fc-a663-aa553b799c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:02.663266 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-43860ca0-be67-41fc-a663-aa553b799c69 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:02.664489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-43860ca0-be67-41fc-a663-aa553b799c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:02.725649 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:02.739325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1119/2234] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:02.751864 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42f428-33d3-4312-b41b-fa1c4e3e6c7c admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:02.753650 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-eb42f428-33d3-4312-b41b-fa1c4e3e6c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:02.824469 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42f428-33d3-4312-b41b-fa1c4e3e6c7c admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:02.828892 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796522e-752d-4ba9-9764-028aacbefa6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:02.829790 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796522e-752d-4ba9-9764-028aacbefa6d service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:02.830082 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796522e-752d-4ba9-9764-028aacbefa6d service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:02.830362 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796522e-752d-4ba9-9764-028aacbefa6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:02.849198 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796522e-752d-4ba9-9764-028aacbefa6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:02.851348 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796522e-752d-4ba9-9764-028aacbefa6d service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:02.866294 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1116/2235] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:41:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:41:03.266562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43860ca0-be67-41fc-a663-aa553b799c69 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:03.268805 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-43860ca0-be67-41fc-a663-aa553b799c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:03.301323 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43860ca0-be67-41fc-a663-aa553b799c69 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:03.308087 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd25a0f-b130-4e5d-bc13-d06fd3e80835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:03.308963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd25a0f-b130-4e5d-bc13-d06fd3e80835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:03.309279 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd25a0f-b130-4e5d-bc13-d06fd3e80835 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:03.309558 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd25a0f-b130-4e5d-bc13-d06fd3e80835 admin admin] PATH_INFO: `/v3/projects/1601306907ce4c3a8ac7abe15538b2e7` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:03.310976 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd25a0f-b130-4e5d-bc13-d06fd3e80835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1601306907ce4c3a8ac7abe15538b2e7)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:03.313313 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd25a0f-b130-4e5d-bc13-d06fd3e80835 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:03.374997 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-4dd25a0f-b130-4e5d-bc13-d06fd3e80835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1601306907ce4c3a8ac7abe15538b2e7', 'request_id': 'req-4dd25a0f-b130-4e5d-bc13-d06fd3e80835'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:03.382578 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-4dd25a0f-b130-4e5d-bc13-d06fd3e80835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1601306907ce4c3a8ac7abe15538b2e7', 'request_id': 'req-4dd25a0f-b130-4e5d-bc13-d06fd3e80835'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:03.389261 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-4dd25a0f-b130-4e5d-bc13-d06fd3e80835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1601306907ce4c3a8ac7abe15538b2e7', 'request_id': 'req-4dd25a0f-b130-4e5d-bc13-d06fd3e80835'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:03.400919 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-4dd25a0f-b130-4e5d-bc13-d06fd3e80835 admin admin] The token cache is being invalidate because project 1601306907ce4c3a8ac7abe15538b2e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:41:03.401551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-4dd25a0f-b130-4e5d-bc13-d06fd3e80835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:03.401899 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-4dd25a0f-b130-4e5d-bc13-d06fd3e80835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:03.403533 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1120/2236] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:41:03 2022] DELETE /identity/v3/projects/1601306907ce4c3a8ac7abe15538b2e7 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:03.409200 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf46914-b9de-4ca4-88fe-973773470eca None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:03.409496 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf46914-b9de-4ca4-88fe-973773470eca None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:03.409881 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf46914-b9de-4ca4-88fe-973773470eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:03.429925 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-0bf46914-b9de-4ca4-88fe-973773470eca None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:03.431438 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-0bf46914-b9de-4ca4-88fe-973773470eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:03.495155 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1117/2237] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:03.502907 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be30aa-668c-40c3-aa58-987499e34536 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:03.503201 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be30aa-668c-40c3-aa58-987499e34536 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:03.503490 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be30aa-668c-40c3-aa58-987499e34536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:03.522628 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-32be30aa-668c-40c3-aa58-987499e34536 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:03.523818 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-32be30aa-668c-40c3-aa58-987499e34536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:03.531849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1121/2238] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:03.536174 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf46914-b9de-4ca4-88fe-973773470eca None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:03.547616 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf46914-b9de-4ca4-88fe-973773470eca None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:03.552541 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb684ec-b1ee-428d-bb59-7fc95bbb965f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:03.553234 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb684ec-b1ee-428d-bb59-7fc95bbb965f admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:03.553516 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb684ec-b1ee-428d-bb59-7fc95bbb965f admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:03.553818 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb684ec-b1ee-428d-bb59-7fc95bbb965f admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:03.554552 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb684ec-b1ee-428d-bb59-7fc95bbb965f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:41:03.554859 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb684ec-b1ee-428d-bb59-7fc95bbb965f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:03.556451 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb684ec-b1ee-428d-bb59-7fc95bbb965f admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:03.574775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1118/2239] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:41:03.576205 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be30aa-668c-40c3-aa58-987499e34536 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:03.585939 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be30aa-668c-40c3-aa58-987499e34536 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:03.590654 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8d3cd-f98c-4b5e-8c81-4ab237b0fd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:03.591336 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8d3cd-f98c-4b5e-8c81-4ab237b0fd7c admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:03.591591 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8d3cd-f98c-4b5e-8c81-4ab237b0fd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:03.591868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8d3cd-f98c-4b5e-8c81-4ab237b0fd7c admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:03.592694 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8d3cd-f98c-4b5e-8c81-4ab237b0fd7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:03.594670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8d3cd-f98c-4b5e-8c81-4ab237b0fd7c admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:03.621520 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1122/2240] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:41:03 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:41:03.625896 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb684ec-b1ee-428d-bb59-7fc95bbb965f admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:03.636740 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb684ec-b1ee-428d-bb59-7fc95bbb965f admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:03.665727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:03.698230 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed051b-dc13-4beb-82d3-0c5fd7abba7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:03.698918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed051b-dc13-4beb-82d3-0c5fd7abba7e admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:03.699189 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed051b-dc13-4beb-82d3-0c5fd7abba7e admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:03.699463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed051b-dc13-4beb-82d3-0c5fd7abba7e admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:03.700193 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed051b-dc13-4beb-82d3-0c5fd7abba7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:03.702170 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed051b-dc13-4beb-82d3-0c5fd7abba7e admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:03.724327 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1119/2241] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:41:03 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:41:03.725495 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8d3cd-f98c-4b5e-8c81-4ab237b0fd7c admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:03.736533 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8d3cd-f98c-4b5e-8c81-4ab237b0fd7c admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:03.741154 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fdb71-9cd9-4669-8b31-f15ae9385436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:03.741813 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fdb71-9cd9-4669-8b31-f15ae9385436 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:03.742093 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fdb71-9cd9-4669-8b31-f15ae9385436 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:03.742368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fdb71-9cd9-4669-8b31-f15ae9385436 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:03.743096 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fdb71-9cd9-4669-8b31-f15ae9385436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:03.745435 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fdb71-9cd9-4669-8b31-f15ae9385436 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:03.755906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1123/2242] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:41:03.760018 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed051b-dc13-4beb-82d3-0c5fd7abba7e admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:03.770068 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed051b-dc13-4beb-82d3-0c5fd7abba7e admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:03.774686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe144ab6-49fa-41b1-9bb1-c7bb1a30e476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:03.775473 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe144ab6-49fa-41b1-9bb1-c7bb1a30e476 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:03.775759 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe144ab6-49fa-41b1-9bb1-c7bb1a30e476 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:03.776041 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe144ab6-49fa-41b1-9bb1-c7bb1a30e476 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:03.776824 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe144ab6-49fa-41b1-9bb1-c7bb1a30e476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:03.778442 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe144ab6-49fa-41b1-9bb1-c7bb1a30e476 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:03.788259 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1120/2243] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:41:03.793024 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fdb71-9cd9-4669-8b31-f15ae9385436 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:03.804026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fdb71-9cd9-4669-8b31-f15ae9385436 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:03.810731 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eac06e-d1ff-4ec2-8e33-98b7c47d217e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:03.811759 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eac06e-d1ff-4ec2-8e33-98b7c47d217e admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:03.812114 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eac06e-d1ff-4ec2-8e33-98b7c47d217e admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:03.812432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eac06e-d1ff-4ec2-8e33-98b7c47d217e admin admin] PATH_INFO: `/v3/projects/f4734eaba7764c0a810861addab506b6/users/c91e80d0ad7248cb88c4630925d20a00/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:03.827842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eac06e-d1ff-4ec2-8e33-98b7c47d217e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4734eaba7764c0a810861addab506b6, user_id=c91e80d0ad7248cb88c4630925d20a00, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:03.830555 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eac06e-d1ff-4ec2-8e33-98b7c47d217e admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:03.843118 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1124/2244] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:41:03 2022] PUT /identity/v3/projects/f4734eaba7764c0a810861addab506b6/users/c91e80d0ad7248cb88c4630925d20a00/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:03.855104 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7b22f-4864-4515-adc1-46befa2c9090 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:03.855396 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7b22f-4864-4515-adc1-46befa2c9090 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:03.855676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7b22f-4864-4515-adc1-46befa2c9090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:03.882866 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-10b7b22f-4864-4515-adc1-46befa2c9090 None None] MFA Rules not processed for user `c91e80d0ad7248cb88c4630925d20a00`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:03.884063 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-10b7b22f-4864-4515-adc1-46befa2c9090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:03.939859 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eac06e-d1ff-4ec2-8e33-98b7c47d217e admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:03.942313 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-00eac06e-d1ff-4ec2-8e33-98b7c47d217e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:03.964493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1121/2245] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:41:03.969275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7b22f-4864-4515-adc1-46befa2c9090 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:04.019911 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eac06e-d1ff-4ec2-8e33-98b7c47d217e admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:04.024987 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7241fd-e846-4e93-a282-09f007bd5254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:04.027082 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7241fd-e846-4e93-a282-09f007bd5254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:04.027082 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7241fd-e846-4e93-a282-09f007bd5254 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:04.027082 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7241fd-e846-4e93-a282-09f007bd5254 admin admin] PATH_INFO: `/v3/users/b63c94bd9e7a488fa9d250a0eb03e185` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:04.028156 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7241fd-e846-4e93-a282-09f007bd5254 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b63c94bd9e7a488fa9d250a0eb03e185)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:04.030221 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7241fd-e846-4e93-a282-09f007bd5254 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:04.038022 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7b22f-4864-4515-adc1-46befa2c9090 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:04.044158 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb79ffc5-8981-4de3-9426-4e33213cc5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:04.044824 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79ffc5-8981-4de3-9426-4e33213cc5e8 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:04.045136 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79ffc5-8981-4de3-9426-4e33213cc5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:04.045420 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79ffc5-8981-4de3-9426-4e33213cc5e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:04.046130 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb79ffc5-8981-4de3-9426-4e33213cc5e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:04.048073 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb79ffc5-8981-4de3-9426-4e33213cc5e8 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:04.061133 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:04.089378 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1122/2246] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:41:03 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:41:04.097600 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb79ffc5-8981-4de3-9426-4e33213cc5e8 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:04.110799 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-ac7241fd-e846-4e93-a282-09f007bd5254 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b63c94bd9e7a488fa9d250a0eb03e185'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:04.123847 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-ac7241fd-e846-4e93-a282-09f007bd5254 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b63c94bd9e7a488fa9d250a0eb03e185'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:04.124594 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-ac7241fd-e846-4e93-a282-09f007bd5254 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b63c94bd9e7a488fa9d250a0eb03e185'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:04.143404 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-ac7241fd-e846-4e93-a282-09f007bd5254 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b63c94bd9e7a488fa9d250a0eb03e185'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:04.144350 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-ac7241fd-e846-4e93-a282-09f007bd5254 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b63c94bd9e7a488fa9d250a0eb03e185'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:04.146019 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:04.184288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1125/2247] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:41:03 2022] DELETE /identity/v3/users/b63c94bd9e7a488fa9d250a0eb03e185 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:41:04.188210 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ff57f-fbed-4145-9805-63d268402a7e None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:04.188508 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ff57f-fbed-4145-9805-63d268402a7e None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:04.189061 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ff57f-fbed-4145-9805-63d268402a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:04.212374 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-518ff57f-fbed-4145-9805-63d268402a7e None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:04.214325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-518ff57f-fbed-4145-9805-63d268402a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:04.231967 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb79ffc5-8981-4de3-9426-4e33213cc5e8 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:04.314506 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1126/2248] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:04.321430 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ff57f-fbed-4145-9805-63d268402a7e None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:04.323828 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-518ff57f-fbed-4145-9805-63d268402a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:04.326733 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550aea31-9786-4fa2-a231-d8d0bcd68f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:04.327524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550aea31-9786-4fa2-a231-d8d0bcd68f95 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:04.327880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550aea31-9786-4fa2-a231-d8d0bcd68f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:04.328232 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550aea31-9786-4fa2-a231-d8d0bcd68f95 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:04.329293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550aea31-9786-4fa2-a231-d8d0bcd68f95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:04.331384 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550aea31-9786-4fa2-a231-d8d0bcd68f95 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:04.375997 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1123/2249] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:41:04 2022] POST /identity/v3/users => generated 289 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:41:04.377940 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550aea31-9786-4fa2-a231-d8d0bcd68f95 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:04.381523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-550aea31-9786-4fa2-a231-d8d0bcd68f95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:04.408235 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550aea31-9786-4fa2-a231-d8d0bcd68f95 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:04.412804 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125009d-7b67-4fc8-8d72-59722660617e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:04.413584 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125009d-7b67-4fc8-8d72-59722660617e admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:04.413937 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125009d-7b67-4fc8-8d72-59722660617e admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:04.414316 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125009d-7b67-4fc8-8d72-59722660617e admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:04.415164 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125009d-7b67-4fc8-8d72-59722660617e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:04.417778 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125009d-7b67-4fc8-8d72-59722660617e admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:04.434826 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1124/2250] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:41:04.438534 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125009d-7b67-4fc8-8d72-59722660617e admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:04.441117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ff57f-fbed-4145-9805-63d268402a7e None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:04.446553 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8712f3-0620-4c24-ba8a-a67ff306b359 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:04.447739 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8712f3-0620-4c24-ba8a-a67ff306b359 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:04.448320 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8712f3-0620-4c24-ba8a-a67ff306b359 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:04.448609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8712f3-0620-4c24-ba8a-a67ff306b359 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:04.459339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125009d-7b67-4fc8-8d72-59722660617e admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:04.460248 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8712f3-0620-4c24-ba8a-a67ff306b359 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:04.461708 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8712f3-0620-4c24-ba8a-a67ff306b359 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:04.464831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d2bcd-9f5b-449d-bb8e-6aa371e6cbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:04.465664 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d2bcd-9f5b-449d-bb8e-6aa371e6cbb2 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:04.466691 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d2bcd-9f5b-449d-bb8e-6aa371e6cbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:04.466691 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d2bcd-9f5b-449d-bb8e-6aa371e6cbb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:04.467386 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d2bcd-9f5b-449d-bb8e-6aa371e6cbb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:04.469740 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d2bcd-9f5b-449d-bb8e-6aa371e6cbb2 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:04.476489 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1127/2251] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:41:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:41:04.489802 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1125/2252] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:41:04.494817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8712f3-0620-4c24-ba8a-a67ff306b359 service neutron] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:04.505821 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8712f3-0620-4c24-ba8a-a67ff306b359 service neutron] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:04.513041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88c107-b1db-41bd-8425-66c4a5f6b22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:04.514004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88c107-b1db-41bd-8425-66c4a5f6b22b admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:04.514281 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88c107-b1db-41bd-8425-66c4a5f6b22b admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:04.514562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88c107-b1db-41bd-8425-66c4a5f6b22b admin admin] PATH_INFO: `/v3/projects/12a0c52727c249f7aa3191d620653394/users/33a5ec2921bc4569a8a183143132a350/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:04.537019 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88c107-b1db-41bd-8425-66c4a5f6b22b admin admin] RBAC: Authorizing `identity:create_grant(project_id=12a0c52727c249f7aa3191d620653394, user_id=33a5ec2921bc4569a8a183143132a350, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:04.541147 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88c107-b1db-41bd-8425-66c4a5f6b22b admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:04.557481 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1128/2253] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:41:04 2022] PUT /identity/v3/projects/12a0c52727c249f7aa3191d620653394/users/33a5ec2921bc4569a8a183143132a350/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:04.628758 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dceb18-4105-4b63-867e-11100bcad19c None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:04.629267 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dceb18-4105-4b63-867e-11100bcad19c None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:04.629374 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dceb18-4105-4b63-867e-11100bcad19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:04.664228 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-23dceb18-4105-4b63-867e-11100bcad19c None None] MFA Rules not processed for user `33a5ec2921bc4569a8a183143132a350`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:04.665949 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-23dceb18-4105-4b63-867e-11100bcad19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:04.759500 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1126/2254] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:04.761557 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88c107-b1db-41bd-8425-66c4a5f6b22b admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:04.781633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:04.832622 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88c107-b1db-41bd-8425-66c4a5f6b22b admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:04.837344 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4124bf-4af9-4086-ae68-4a5b6351b0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:04.838015 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4124bf-4af9-4086-ae68-4a5b6351b0da admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:04.838316 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4124bf-4af9-4086-ae68-4a5b6351b0da admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:04.838573 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4124bf-4af9-4086-ae68-4a5b6351b0da admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:04.839265 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4124bf-4af9-4086-ae68-4a5b6351b0da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:04.841216 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4124bf-4af9-4086-ae68-4a5b6351b0da admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:04.867702 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1129/2255] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:41:04 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:41:04.872165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dceb18-4105-4b63-867e-11100bcad19c None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:04.883653 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dceb18-4105-4b63-867e-11100bcad19c None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:04.916861 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:04.948252 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4124bf-4af9-4086-ae68-4a5b6351b0da admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:04.950324 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-be4124bf-4af9-4086-ae68-4a5b6351b0da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:04.959310 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43567553-a093-426f-a4f6-5c6f39843e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:04.960002 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43567553-a093-426f-a4f6-5c6f39843e6b admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:04.960280 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43567553-a093-426f-a4f6-5c6f39843e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:04.960556 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43567553-a093-426f-a4f6-5c6f39843e6b admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:04.961346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43567553-a093-426f-a4f6-5c6f39843e6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:04.963535 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43567553-a093-426f-a4f6-5c6f39843e6b admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:04.996398 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1127/2256] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:41:04 2022] POST /identity/v3/users => generated 294 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:41:05.000059 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43567553-a093-426f-a4f6-5c6f39843e6b admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:05.011175 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43567553-a093-426f-a4f6-5c6f39843e6b admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:05.023277 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400f068-db48-43ec-82e9-19f4cf8820d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:05.026815 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400f068-db48-43ec-82e9-19f4cf8820d0 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:05.027262 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400f068-db48-43ec-82e9-19f4cf8820d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:05.028296 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400f068-db48-43ec-82e9-19f4cf8820d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:05.028618 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400f068-db48-43ec-82e9-19f4cf8820d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:05.030444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400f068-db48-43ec-82e9-19f4cf8820d0 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:05.049114 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1128/2257] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:41:05.049340 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4124bf-4af9-4086-ae68-4a5b6351b0da admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:05.050811 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400f068-db48-43ec-82e9-19f4cf8820d0 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:05.052231 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c0641-1e0b-43a2-a18e-b04aeb0711c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:05.053952 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c0641-1e0b-43a2-a18e-b04aeb0711c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:05.054241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c0641-1e0b-43a2-a18e-b04aeb0711c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:05.054517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c0641-1e0b-43a2-a18e-b04aeb0711c0 admin admin] PATH_INFO: `/v3/projects/7ed6e7435e2f4e2484eca25601ced86c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:05.055853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c0641-1e0b-43a2-a18e-b04aeb0711c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ed6e7435e2f4e2484eca25601ced86c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:05.058528 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c0641-1e0b-43a2-a18e-b04aeb0711c0 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:05.063907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400f068-db48-43ec-82e9-19f4cf8820d0 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:05.069496 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579317de-13e2-4cb8-aa5a-fab72d916cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:05.070661 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579317de-13e2-4cb8-aa5a-fab72d916cf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:05.071462 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579317de-13e2-4cb8-aa5a-fab72d916cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:05.071753 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579317de-13e2-4cb8-aa5a-fab72d916cf9 admin admin] PATH_INFO: `/v3/projects/24e862ba00e2495db17f66ca54849696/users/b6252a52cd8a49c3accc8ecd1764a336/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:05.098594 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579317de-13e2-4cb8-aa5a-fab72d916cf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24e862ba00e2495db17f66ca54849696, user_id=b6252a52cd8a49c3accc8ecd1764a336, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:05.101483 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579317de-13e2-4cb8-aa5a-fab72d916cf9 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:05.117729 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1129/2258] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:41:05 2022] PUT /identity/v3/projects/24e862ba00e2495db17f66ca54849696/users/b6252a52cd8a49c3accc8ecd1764a336/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:05.122245 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579317de-13e2-4cb8-aa5a-fab72d916cf9 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:05.125160 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-804c0641-1e0b-43a2-a18e-b04aeb0711c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ed6e7435e2f4e2484eca25601ced86c', 'request_id': 'req-804c0641-1e0b-43a2-a18e-b04aeb0711c0'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:05.134765 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-804c0641-1e0b-43a2-a18e-b04aeb0711c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ed6e7435e2f4e2484eca25601ced86c', 'request_id': 'req-804c0641-1e0b-43a2-a18e-b04aeb0711c0'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:05.141885 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-804c0641-1e0b-43a2-a18e-b04aeb0711c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ed6e7435e2f4e2484eca25601ced86c', 'request_id': 'req-804c0641-1e0b-43a2-a18e-b04aeb0711c0'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:05.158320 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-804c0641-1e0b-43a2-a18e-b04aeb0711c0 admin admin] The token cache is being invalidate because project 7ed6e7435e2f4e2484eca25601ced86c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:41:05.158616 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-804c0641-1e0b-43a2-a18e-b04aeb0711c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:05.159311 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-804c0641-1e0b-43a2-a18e-b04aeb0711c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:05.160858 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1130/2259] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:41:04 2022] DELETE /identity/v3/projects/7ed6e7435e2f4e2484eca25601ced86c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:05.166833 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d86be3-c7e5-4c16-857c-91de50d5a9ac None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:05.167273 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d86be3-c7e5-4c16-857c-91de50d5a9ac None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:05.167745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d86be3-c7e5-4c16-857c-91de50d5a9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:05.190448 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-12d86be3-c7e5-4c16-857c-91de50d5a9ac None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:05.191954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-12d86be3-c7e5-4c16-857c-91de50d5a9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:05.207394 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579317de-13e2-4cb8-aa5a-fab72d916cf9 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:05.212660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783c935-3195-43f6-aeb2-0fe90a79a66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:05.213873 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783c935-3195-43f6-aeb2-0fe90a79a66c admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:05.214201 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783c935-3195-43f6-aeb2-0fe90a79a66c admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:05.214701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783c935-3195-43f6-aeb2-0fe90a79a66c admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:05.215349 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783c935-3195-43f6-aeb2-0fe90a79a66c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:05.217054 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783c935-3195-43f6-aeb2-0fe90a79a66c admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:05.227359 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:05.228927 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1130/2260] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:41:05.233223 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783c935-3195-43f6-aeb2-0fe90a79a66c admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:05.242561 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-7783c935-3195-43f6-aeb2-0fe90a79a66c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:05.278404 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783c935-3195-43f6-aeb2-0fe90a79a66c admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:05.283161 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1131/2261] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:41:05.292945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9f5df-8e6b-459e-acee-6ef8fd95ed6f None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:05.293176 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9f5df-8e6b-459e-acee-6ef8fd95ed6f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:05.293486 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9f5df-8e6b-459e-acee-6ef8fd95ed6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:05.300206 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a988a-c479-4de5-be24-e5176517541b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:05.300206 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a988a-c479-4de5-be24-e5176517541b admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:05.300206 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a988a-c479-4de5-be24-e5176517541b admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:05.300712 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a988a-c479-4de5-be24-e5176517541b admin admin] PATH_INFO: `/v3/projects/24e862ba00e2495db17f66ca54849696/users/b6252a52cd8a49c3accc8ecd1764a336/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:05.303484 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a988a-c479-4de5-be24-e5176517541b admin admin] RBAC: Authorizing `identity:create_grant(project_id=24e862ba00e2495db17f66ca54849696, user_id=b6252a52cd8a49c3accc8ecd1764a336, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:05.306283 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a988a-c479-4de5-be24-e5176517541b admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:05.318332 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1131/2262] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:41:05 2022] PUT /identity/v3/projects/24e862ba00e2495db17f66ca54849696/users/b6252a52cd8a49c3accc8ecd1764a336/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:41:05.331574 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41317f08-1188-472a-88d8-df6d7c760d7e None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:05.332011 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41317f08-1188-472a-88d8-df6d7c760d7e None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:05.332293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41317f08-1188-472a-88d8-df6d7c760d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:05.335543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-9cc9f5df-8e6b-459e-acee-6ef8fd95ed6f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:05.336839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-9cc9f5df-8e6b-459e-acee-6ef8fd95ed6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:05.377576 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-41317f08-1188-472a-88d8-df6d7c760d7e None None] MFA Rules not processed for user `b6252a52cd8a49c3accc8ecd1764a336`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:05.378867 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-41317f08-1188-472a-88d8-df6d7c760d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:05.405464 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1132/2263] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:05.410006 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9f5df-8e6b-459e-acee-6ef8fd95ed6f None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:05.419754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9f5df-8e6b-459e-acee-6ef8fd95ed6f None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:05.424434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6da65e-073f-436b-82d6-bd966b5a9758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:05.425098 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6da65e-073f-436b-82d6-bd966b5a9758 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:05.425387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6da65e-073f-436b-82d6-bd966b5a9758 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:05.425659 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6da65e-073f-436b-82d6-bd966b5a9758 admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:05.426392 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6da65e-073f-436b-82d6-bd966b5a9758 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:41:05.426698 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6da65e-073f-436b-82d6-bd966b5a9758 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:05.428298 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6da65e-073f-436b-82d6-bd966b5a9758 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:05.434156 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:05.441115 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1133/2264] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:41:05.444493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6da65e-073f-436b-82d6-bd966b5a9758 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:05.454688 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6da65e-073f-436b-82d6-bd966b5a9758 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:05.458274 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1132/2265] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:41:05.460841 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7f2e0-14ae-44e9-9d01-de894db5352a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:05.461751 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7f2e0-14ae-44e9-9d01-de894db5352a admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:05.462175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7f2e0-14ae-44e9-9d01-de894db5352a admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:05.462453 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7f2e0-14ae-44e9-9d01-de894db5352a admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:05.463802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7f2e0-14ae-44e9-9d01-de894db5352a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:05.465749 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7f2e0-14ae-44e9-9d01-de894db5352a admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:05.471555 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41317f08-1188-472a-88d8-df6d7c760d7e None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:05.478512 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-41317f08-1188-472a-88d8-df6d7c760d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:05.507451 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1134/2266] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:41:05 2022] POST /identity/v3/projects => generated 363 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:41:05.512696 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7f2e0-14ae-44e9-9d01-de894db5352a admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:05.552559 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41317f08-1188-472a-88d8-df6d7c760d7e None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:05.606966 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7f2e0-14ae-44e9-9d01-de894db5352a admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:05.612782 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc2ef6-18e8-46bf-b0fb-9c4904dbe86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:05.613544 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc2ef6-18e8-46bf-b0fb-9c4904dbe86c admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:05.613891 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc2ef6-18e8-46bf-b0fb-9c4904dbe86c admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:05.614264 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc2ef6-18e8-46bf-b0fb-9c4904dbe86c admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:05.615032 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc2ef6-18e8-46bf-b0fb-9c4904dbe86c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:05.615153 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53dae2e-4e29-45d4-a222-a840aa779330 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:05.615868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53dae2e-4e29-45d4-a222-a840aa779330 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:05.616229 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53dae2e-4e29-45d4-a222-a840aa779330 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:05.616578 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53dae2e-4e29-45d4-a222-a840aa779330 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:05.617304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc2ef6-18e8-46bf-b0fb-9c4904dbe86c admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:05.618748 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-b53dae2e-4e29-45d4-a222-a840aa779330 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:05.665397 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1135/2267] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:41:05 2022] POST /identity/v3/users => generated 282 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:41:05.670104 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc2ef6-18e8-46bf-b0fb-9c4904dbe86c admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:05.688698 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc2ef6-18e8-46bf-b0fb-9c4904dbe86c admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:05.694450 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3e76e-1e2d-486d-89bd-6534888e23db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:05.695234 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3e76e-1e2d-486d-89bd-6534888e23db admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:05.695575 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3e76e-1e2d-486d-89bd-6534888e23db admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:05.695847 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3e76e-1e2d-486d-89bd-6534888e23db admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:05.696633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3e76e-1e2d-486d-89bd-6534888e23db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:05.698386 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3e76e-1e2d-486d-89bd-6534888e23db admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:05.711847 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53dae2e-4e29-45d4-a222-a840aa779330 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:05.716022 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1136/2268] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:41:05.716022 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3e76e-1e2d-486d-89bd-6534888e23db admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:05.716194 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53dae2e-4e29-45d4-a222-a840aa779330 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:05.716301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-efb3e76e-1e2d-486d-89bd-6534888e23db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:05.725355 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1133/2269] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:41:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:41:05.726216 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53dae2e-4e29-45d4-a222-a840aa779330 service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:05.730760 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3e76e-1e2d-486d-89bd-6534888e23db admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:05.737515 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf4011-0b33-4865-8c96-7b7842b6f815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:05.738666 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf4011-0b33-4865-8c96-7b7842b6f815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:05.740213 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf4011-0b33-4865-8c96-7b7842b6f815 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:05.740503 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf4011-0b33-4865-8c96-7b7842b6f815 admin admin] PATH_INFO: `/v3/users/d933963e196b4579bfa3abe2d1a5b919` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:05.741094 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53dae2e-4e29-45d4-a222-a840aa779330 service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:05.742530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf4011-0b33-4865-8c96-7b7842b6f815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d933963e196b4579bfa3abe2d1a5b919)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:05.744836 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf4011-0b33-4865-8c96-7b7842b6f815 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:05.746370 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959037d1-558e-4f0c-87dd-28ce6116cf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:05.747090 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959037d1-558e-4f0c-87dd-28ce6116cf8a admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:05.747377 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959037d1-558e-4f0c-87dd-28ce6116cf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:05.747660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959037d1-558e-4f0c-87dd-28ce6116cf8a admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:05.748454 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959037d1-558e-4f0c-87dd-28ce6116cf8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:05.750086 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959037d1-558e-4f0c-87dd-28ce6116cf8a admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:05.759234 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:05.762304 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1134/2270] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:41:05.766945 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959037d1-558e-4f0c-87dd-28ce6116cf8a admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:05.780612 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959037d1-558e-4f0c-87dd-28ce6116cf8a admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:05.788550 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a5a1f-6d8b-4a62-912d-6eec029e10c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:05.789599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a5a1f-6d8b-4a62-912d-6eec029e10c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:05.790068 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a5a1f-6d8b-4a62-912d-6eec029e10c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:05.790360 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a5a1f-6d8b-4a62-912d-6eec029e10c2 admin admin] PATH_INFO: `/v3/projects/c59efd8a923f4b67b7fe8f6478cffe95/users/bc8b20419fcf4269a24ff5b6f625d54d/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:05.809389 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-8ccf4011-0b33-4865-8c96-7b7842b6f815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd933963e196b4579bfa3abe2d1a5b919'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:05.817645 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a5a1f-6d8b-4a62-912d-6eec029e10c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c59efd8a923f4b67b7fe8f6478cffe95, user_id=bc8b20419fcf4269a24ff5b6f625d54d, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:05.820638 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a5a1f-6d8b-4a62-912d-6eec029e10c2 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:05.823518 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-8ccf4011-0b33-4865-8c96-7b7842b6f815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd933963e196b4579bfa3abe2d1a5b919'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:05.825399 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-8ccf4011-0b33-4865-8c96-7b7842b6f815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd933963e196b4579bfa3abe2d1a5b919'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:05.841233 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1135/2271] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:41:05 2022] PUT /identity/v3/projects/c59efd8a923f4b67b7fe8f6478cffe95/users/bc8b20419fcf4269a24ff5b6f625d54d/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:05.845188 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-8ccf4011-0b33-4865-8c96-7b7842b6f815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd933963e196b4579bfa3abe2d1a5b919'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:05.846805 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-8ccf4011-0b33-4865-8c96-7b7842b6f815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd933963e196b4579bfa3abe2d1a5b919'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:05.856662 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba6968-04a7-4e0a-97ce-30691ded8609 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:05.856662 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba6968-04a7-4e0a-97ce-30691ded8609 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:05.856662 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba6968-04a7-4e0a-97ce-30691ded8609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:05.886170 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1137/2272] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:41:05 2022] DELETE /identity/v3/users/d933963e196b4579bfa3abe2d1a5b919 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:05.889281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-4cba6968-04a7-4e0a-97ce-30691ded8609 None None] MFA Rules not processed for user `bc8b20419fcf4269a24ff5b6f625d54d`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:05.889783 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-4cba6968-04a7-4e0a-97ce-30691ded8609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:05.891377 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1934c2e-1417-4b3d-9007-a2ee69fcd308 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:05.891667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1934c2e-1417-4b3d-9007-a2ee69fcd308 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:05.891945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1934c2e-1417-4b3d-9007-a2ee69fcd308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:05.914533 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:05.924307 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-a1934c2e-1417-4b3d-9007-a2ee69fcd308 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:05.925620 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a1934c2e-1417-4b3d-9007-a2ee69fcd308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:05.993616 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1136/2273] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:05 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:41:05.999969 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1138/2274] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:06.006137 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba6968-04a7-4e0a-97ce-30691ded8609 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:06.009011 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-4cba6968-04a7-4e0a-97ce-30691ded8609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:06.010232 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1934c2e-1417-4b3d-9007-a2ee69fcd308 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:06.011633 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a1934c2e-1417-4b3d-9007-a2ee69fcd308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:06.084178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba6968-04a7-4e0a-97ce-30691ded8609 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:06.088433 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1934c2e-1417-4b3d-9007-a2ee69fcd308 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:06.093283 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062e384-a479-40f7-9b8f-a081aa4bc5c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:06.095941 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062e384-a479-40f7-9b8f-a081aa4bc5c5 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:06.096205 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062e384-a479-40f7-9b8f-a081aa4bc5c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:06.096480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062e384-a479-40f7-9b8f-a081aa4bc5c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:06.097863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b1fe9-8375-4dfc-96c1-1b1dd490e3f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:06.098562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b1fe9-8375-4dfc-96c1-1b1dd490e3f8 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:06.098850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b1fe9-8375-4dfc-96c1-1b1dd490e3f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:06.099183 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b1fe9-8375-4dfc-96c1-1b1dd490e3f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:06.110868 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062e384-a479-40f7-9b8f-a081aa4bc5c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:06.112594 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062e384-a479-40f7-9b8f-a081aa4bc5c5 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:06.116677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b1fe9-8375-4dfc-96c1-1b1dd490e3f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:06.119941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b1fe9-8375-4dfc-96c1-1b1dd490e3f8 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:06.127606 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1139/2275] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:41:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:41:06.161141 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1137/2276] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:41:06 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:41:06.535467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062e384-a479-40f7-9b8f-a081aa4bc5c5 service neutron] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:06.536894 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-f062e384-a479-40f7-9b8f-a081aa4bc5c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:06.558326 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062e384-a479-40f7-9b8f-a081aa4bc5c5 service neutron] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:06.563043 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e5e32-eb9a-4ab6-b0a4-f4377f2afd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:06.564161 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e5e32-eb9a-4ab6-b0a4-f4377f2afd7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:06.564544 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e5e32-eb9a-4ab6-b0a4-f4377f2afd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:06.564920 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e5e32-eb9a-4ab6-b0a4-f4377f2afd7b admin admin] PATH_INFO: `/v3/projects/ebac0198d5f74999b1ac864de62b13f7` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:06.566306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e5e32-eb9a-4ab6-b0a4-f4377f2afd7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebac0198d5f74999b1ac864de62b13f7)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:06.568513 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e5e32-eb9a-4ab6-b0a4-f4377f2afd7b admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:06.611062 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:06.637626 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-944e5e32-eb9a-4ab6-b0a4-f4377f2afd7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebac0198d5f74999b1ac864de62b13f7', 'request_id': 'req-944e5e32-eb9a-4ab6-b0a4-f4377f2afd7b'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:06.646791 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-944e5e32-eb9a-4ab6-b0a4-f4377f2afd7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebac0198d5f74999b1ac864de62b13f7', 'request_id': 'req-944e5e32-eb9a-4ab6-b0a4-f4377f2afd7b'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:06.653905 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-944e5e32-eb9a-4ab6-b0a4-f4377f2afd7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebac0198d5f74999b1ac864de62b13f7', 'request_id': 'req-944e5e32-eb9a-4ab6-b0a4-f4377f2afd7b'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:06.669009 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-944e5e32-eb9a-4ab6-b0a4-f4377f2afd7b admin admin] The token cache is being invalidate because project ebac0198d5f74999b1ac864de62b13f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:41:06.669352 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-944e5e32-eb9a-4ab6-b0a4-f4377f2afd7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:06.670140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-944e5e32-eb9a-4ab6-b0a4-f4377f2afd7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:06.671750 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1140/2277] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:41:06 2022] DELETE /identity/v3/projects/ebac0198d5f74999b1ac864de62b13f7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:06.675826 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b1fe9-8375-4dfc-96c1-1b1dd490e3f8 service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:06.687354 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b1fe9-8375-4dfc-96c1-1b1dd490e3f8 service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:06.762645 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9668a-8b15-41b1-952c-31745d44dc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:06.763671 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9668a-8b15-41b1-952c-31745d44dc4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:06.763956 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9668a-8b15-41b1-952c-31745d44dc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:06.764232 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9668a-8b15-41b1-952c-31745d44dc4a admin admin] PATH_INFO: `/v3/users/675ef75ee32c4bb48263fa07524875a1` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:06.765945 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e9668a-8b15-41b1-952c-31745d44dc4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=675ef75ee32c4bb48263fa07524875a1)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:06.767900 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e9668a-8b15-41b1-952c-31745d44dc4a admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:06.827778 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-a3e9668a-8b15-41b1-952c-31745d44dc4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '675ef75ee32c4bb48263fa07524875a1'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:06.835028 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-a3e9668a-8b15-41b1-952c-31745d44dc4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '675ef75ee32c4bb48263fa07524875a1'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:06.835660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-a3e9668a-8b15-41b1-952c-31745d44dc4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '675ef75ee32c4bb48263fa07524875a1'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:06.843725 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:06.855323 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-a3e9668a-8b15-41b1-952c-31745d44dc4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '675ef75ee32c4bb48263fa07524875a1'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:06.865602 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-a3e9668a-8b15-41b1-952c-31745d44dc4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '675ef75ee32c4bb48263fa07524875a1'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:06.873789 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1138/2278] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:41:06 2022] DELETE /identity/v3/users/675ef75ee32c4bb48263fa07524875a1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:41:07.306870 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e5e32-eb9a-4ab6-b0a4-f4377f2afd7b admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:07.315465 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-944e5e32-eb9a-4ab6-b0a4-f4377f2afd7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:07.348681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e5e32-eb9a-4ab6-b0a4-f4377f2afd7b admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:07.416574 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9668a-8b15-41b1-952c-31745d44dc4a admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:07.418282 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-a3e9668a-8b15-41b1-952c-31745d44dc4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:07.425137 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8a42d-43a9-4613-a6e1-43c1d279cf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:07.426019 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8a42d-43a9-4613-a6e1-43c1d279cf23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:07.426307 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8a42d-43a9-4613-a6e1-43c1d279cf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:07.426652 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8a42d-43a9-4613-a6e1-43c1d279cf23 admin admin] PATH_INFO: `/v3/projects/161fdfe0f78b4aafba1b0d16df8b4c64` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:07.428082 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8a42d-43a9-4613-a6e1-43c1d279cf23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=161fdfe0f78b4aafba1b0d16df8b4c64)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:07.430903 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8a42d-43a9-4613-a6e1-43c1d279cf23 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:07.493122 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9668a-8b15-41b1-952c-31745d44dc4a admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:07.508909 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-33d8a42d-43a9-4613-a6e1-43c1d279cf23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '161fdfe0f78b4aafba1b0d16df8b4c64', 'request_id': 'req-33d8a42d-43a9-4613-a6e1-43c1d279cf23'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:07.524266 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-33d8a42d-43a9-4613-a6e1-43c1d279cf23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '161fdfe0f78b4aafba1b0d16df8b4c64', 'request_id': 'req-33d8a42d-43a9-4613-a6e1-43c1d279cf23'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:07.533951 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-33d8a42d-43a9-4613-a6e1-43c1d279cf23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '161fdfe0f78b4aafba1b0d16df8b4c64', 'request_id': 'req-33d8a42d-43a9-4613-a6e1-43c1d279cf23'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:07.549857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-33d8a42d-43a9-4613-a6e1-43c1d279cf23 admin admin] The token cache is being invalidate because project 161fdfe0f78b4aafba1b0d16df8b4c64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:41:07.550548 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-33d8a42d-43a9-4613-a6e1-43c1d279cf23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:07.551517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-33d8a42d-43a9-4613-a6e1-43c1d279cf23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:07.557960 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1141/2279] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:41:07 2022] DELETE /identity/v3/projects/161fdfe0f78b4aafba1b0d16df8b4c64 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:07.559309 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e159720-e296-40fe-8bec-c580e20969fa None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:07.559593 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e159720-e296-40fe-8bec-c580e20969fa None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:07.559925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e159720-e296-40fe-8bec-c580e20969fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:07.563850 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2baa240-bcbd-4dc0-a613-0918f7798d33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:07.564676 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2baa240-bcbd-4dc0-a613-0918f7798d33 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:07.565029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2baa240-bcbd-4dc0-a613-0918f7798d33 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:07.565415 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2baa240-bcbd-4dc0-a613-0918f7798d33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:07.567525 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-e2baa240-bcbd-4dc0-a613-0918f7798d33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:07.580859 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-8e159720-e296-40fe-8bec-c580e20969fa None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:07.582615 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-8e159720-e296-40fe-8bec-c580e20969fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:07.671484 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:07.672281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2baa240-bcbd-4dc0-a613-0918f7798d33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:07.674145 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2baa240-bcbd-4dc0-a613-0918f7798d33 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:07.689393 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1139/2280] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:41:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:41:07.695250 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1142/2281] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:07.711026 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77c480-6049-4961-9125-6b492945e1df None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:07.711321 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77c480-6049-4961-9125-6b492945e1df None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:07.711605 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77c480-6049-4961-9125-6b492945e1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:07.733024 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-0f77c480-6049-4961-9125-6b492945e1df None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:07.734335 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-0f77c480-6049-4961-9125-6b492945e1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:07.751857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1140/2282] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:07.757799 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e159720-e296-40fe-8bec-c580e20969fa None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:07.766611 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e159720-e296-40fe-8bec-c580e20969fa None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:07.772099 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad44529-e623-4972-befd-5e2a35416840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:07.772838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad44529-e623-4972-befd-5e2a35416840 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:07.773241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad44529-e623-4972-befd-5e2a35416840 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:07.773551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad44529-e623-4972-befd-5e2a35416840 admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:07.774378 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad44529-e623-4972-befd-5e2a35416840 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:41:07.774692 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad44529-e623-4972-befd-5e2a35416840 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:07.777143 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad44529-e623-4972-befd-5e2a35416840 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:07.792383 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1143/2283] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:41:07.799101 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77c480-6049-4961-9125-6b492945e1df None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:07.810116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77c480-6049-4961-9125-6b492945e1df None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:07.817246 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b91c0-a5e7-4f08-b07e-bd87e6fd41ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:07.818011 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b91c0-a5e7-4f08-b07e-bd87e6fd41ce admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:07.818308 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b91c0-a5e7-4f08-b07e-bd87e6fd41ce admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:07.818583 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b91c0-a5e7-4f08-b07e-bd87e6fd41ce admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:07.821426 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b91c0-a5e7-4f08-b07e-bd87e6fd41ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:07.821426 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b91c0-a5e7-4f08-b07e-bd87e6fd41ce admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:07.869335 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1141/2284] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:41:07 2022] POST /identity/v3/projects => generated 391 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:41:07.871232 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad44529-e623-4972-befd-5e2a35416840 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:07.883289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad44529-e623-4972-befd-5e2a35416840 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:07.988738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec46414-ee0c-4787-b74f-800c8b82d1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:07.992262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec46414-ee0c-4787-b74f-800c8b82d1d5 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:07.992617 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec46414-ee0c-4787-b74f-800c8b82d1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:07.993467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec46414-ee0c-4787-b74f-800c8b82d1d5 admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:07.994207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec46414-ee0c-4787-b74f-800c8b82d1d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:07.998150 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec46414-ee0c-4787-b74f-800c8b82d1d5 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:08.035510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1144/2285] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:41:07 2022] POST /identity/v3/users => generated 296 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:41:08.039374 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b91c0-a5e7-4f08-b07e-bd87e6fd41ce admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:08.057153 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b91c0-a5e7-4f08-b07e-bd87e6fd41ce admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:08.070626 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8a128-6033-42ff-958d-d9569de8eede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:08.072836 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8a128-6033-42ff-958d-d9569de8eede admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:08.072836 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8a128-6033-42ff-958d-d9569de8eede admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:08.072836 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8a128-6033-42ff-958d-d9569de8eede admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:08.074219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8a128-6033-42ff-958d-d9569de8eede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:08.076400 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8a128-6033-42ff-958d-d9569de8eede admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:08.101512 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1142/2286] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:41:08.105915 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec46414-ee0c-4787-b74f-800c8b82d1d5 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:08.116700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec46414-ee0c-4787-b74f-800c8b82d1d5 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:08.121732 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09373d7f-82d7-4137-8d23-6c25977492cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:08.122500 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09373d7f-82d7-4137-8d23-6c25977492cd admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:08.123051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09373d7f-82d7-4137-8d23-6c25977492cd admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:08.123408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09373d7f-82d7-4137-8d23-6c25977492cd admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:08.124305 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09373d7f-82d7-4137-8d23-6c25977492cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:08.126042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09373d7f-82d7-4137-8d23-6c25977492cd admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:08.137647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1145/2287] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:41:08.142739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8a128-6033-42ff-958d-d9569de8eede admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:08.157466 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8a128-6033-42ff-958d-d9569de8eede admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:08.165821 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ac849-3efd-4588-9c27-1a2577177cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:08.166990 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ac849-3efd-4588-9c27-1a2577177cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:08.167284 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ac849-3efd-4588-9c27-1a2577177cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:08.167563 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ac849-3efd-4588-9c27-1a2577177cf4 admin admin] PATH_INFO: `/v3/projects/adb4d921f0fc4f1487a0bac99b59ebb3/users/6ffaec80a3e94f498e1258a01dd986e9/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:08.170972 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:08.186145 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ac849-3efd-4588-9c27-1a2577177cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adb4d921f0fc4f1487a0bac99b59ebb3, user_id=6ffaec80a3e94f498e1258a01dd986e9, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:08.188947 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ac849-3efd-4588-9c27-1a2577177cf4 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:08.203811 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1143/2288] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:41:08 2022] PUT /identity/v3/projects/adb4d921f0fc4f1487a0bac99b59ebb3/users/6ffaec80a3e94f498e1258a01dd986e9/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:41:08.215743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cba4b-5831-4c34-ac81-a5cc5bc5e486 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:08.216041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cba4b-5831-4c34-ac81-a5cc5bc5e486 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:08.216325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cba4b-5831-4c34-ac81-a5cc5bc5e486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:08.248175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-8b6cba4b-5831-4c34-ac81-a5cc5bc5e486 None None] MFA Rules not processed for user `6ffaec80a3e94f498e1258a01dd986e9`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:08.249692 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-8b6cba4b-5831-4c34-ac81-a5cc5bc5e486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:08.339807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1146/2289] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:08.341326 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ac849-3efd-4588-9c27-1a2577177cf4 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:08.466095 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ac849-3efd-4588-9c27-1a2577177cf4 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:08.487009 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cdbdf-c836-49c7-9fd5-ecad5796bd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:08.487981 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cdbdf-c836-49c7-9fd5-ecad5796bd99 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:08.488355 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cdbdf-c836-49c7-9fd5-ecad5796bd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:08.488789 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cdbdf-c836-49c7-9fd5-ecad5796bd99 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:08.489985 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017cdbdf-c836-49c7-9fd5-ecad5796bd99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:08.492157 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017cdbdf-c836-49c7-9fd5-ecad5796bd99 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:08.520769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1144/2290] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:41:08 2022] POST /identity/v3/projects => generated 391 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:41:08.525432 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cba4b-5831-4c34-ac81-a5cc5bc5e486 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:08.537052 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cba4b-5831-4c34-ac81-a5cc5bc5e486 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:08.558860 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cdbdf-c836-49c7-9fd5-ecad5796bd99 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:08.560564 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-017cdbdf-c836-49c7-9fd5-ecad5796bd99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:08.581964 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:08.602029 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9521f-58a6-4fb7-b27a-fdeb4f8abcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:08.602849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9521f-58a6-4fb7-b27a-fdeb4f8abcd7 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:08.603203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9521f-58a6-4fb7-b27a-fdeb4f8abcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:08.603593 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9521f-58a6-4fb7-b27a-fdeb4f8abcd7 admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:08.604396 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9521f-58a6-4fb7-b27a-fdeb4f8abcd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:08.606612 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9521f-58a6-4fb7-b27a-fdeb4f8abcd7 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:08.623219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cdbdf-c836-49c7-9fd5-ecad5796bd99 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:08.626831 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecd79c-c748-43ce-9f16-c6a9a80d33ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:08.627458 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecd79c-c748-43ce-9f16-c6a9a80d33ea service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:08.627734 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecd79c-c748-43ce-9f16-c6a9a80d33ea service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:08.628009 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecd79c-c748-43ce-9f16-c6a9a80d33ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:08.628160 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1147/2291] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:41:08 2022] POST /identity/v3/users => generated 302 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:41:08.629852 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-daecd79c-c748-43ce-9f16-c6a9a80d33ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:08.632290 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9521f-58a6-4fb7-b27a-fdeb4f8abcd7 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:08.642946 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9521f-58a6-4fb7-b27a-fdeb4f8abcd7 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:08.647616 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c65595-d72c-4955-bed7-c99bdb465d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:08.648268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c65595-d72c-4955-bed7-c99bdb465d73 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:08.648557 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c65595-d72c-4955-bed7-c99bdb465d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:08.648837 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c65595-d72c-4955-bed7-c99bdb465d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:08.649609 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c65595-d72c-4955-bed7-c99bdb465d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:08.653617 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c65595-d72c-4955-bed7-c99bdb465d73 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:08.665122 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1148/2292] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:41:08.668204 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c65595-d72c-4955-bed7-c99bdb465d73 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:08.678853 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c65595-d72c-4955-bed7-c99bdb465d73 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:08.685994 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f3f7e-9501-4532-bba9-0487ffcac86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:08.686879 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f3f7e-9501-4532-bba9-0487ffcac86b admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:08.687161 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f3f7e-9501-4532-bba9-0487ffcac86b admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:08.687437 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f3f7e-9501-4532-bba9-0487ffcac86b admin admin] PATH_INFO: `/v3/projects/2da4da2da2da4540bec3d441d7cf8ca9/users/fd5f4301e59047acbfb758a1fd772d03/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:08.707602 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daecd79c-c748-43ce-9f16-c6a9a80d33ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:08.707842 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f3f7e-9501-4532-bba9-0487ffcac86b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da4da2da2da4540bec3d441d7cf8ca9, user_id=fd5f4301e59047acbfb758a1fd772d03, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:08.709110 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daecd79c-c748-43ce-9f16-c6a9a80d33ea service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:08.710757 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f3f7e-9501-4532-bba9-0487ffcac86b admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:08.721729 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1145/2293] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:41:08 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:41:08.728017 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1149/2294] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:41:08 2022] PUT /identity/v3/projects/2da4da2da2da4540bec3d441d7cf8ca9/users/fd5f4301e59047acbfb758a1fd772d03/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:08.731771 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecd79c-c748-43ce-9f16-c6a9a80d33ea service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:08.742478 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecd79c-c748-43ce-9f16-c6a9a80d33ea service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:08.761825 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:08.816667 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a8952-fa0a-427c-8c2b-1d8428f0508a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:08.817286 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a8952-fa0a-427c-8c2b-1d8428f0508a admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:08.817286 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a8952-fa0a-427c-8c2b-1d8428f0508a admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:08.817286 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a8952-fa0a-427c-8c2b-1d8428f0508a admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:08.818260 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a8952-fa0a-427c-8c2b-1d8428f0508a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:08.820536 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a8952-fa0a-427c-8c2b-1d8428f0508a admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:08.831993 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1146/2295] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:41:08.836514 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f3f7e-9501-4532-bba9-0487ffcac86b admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:08.849523 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f3f7e-9501-4532-bba9-0487ffcac86b admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:08.854832 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2aa44-ea65-44ea-80a1-b32f57e769a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:08.855765 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2aa44-ea65-44ea-80a1-b32f57e769a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:08.856043 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2aa44-ea65-44ea-80a1-b32f57e769a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:08.856315 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2aa44-ea65-44ea-80a1-b32f57e769a3 admin admin] PATH_INFO: `/v3/projects/2da4da2da2da4540bec3d441d7cf8ca9/users/fd5f4301e59047acbfb758a1fd772d03/roles/19f5e72c0be44a1197c7288de82e79e8` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:08.858315 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2aa44-ea65-44ea-80a1-b32f57e769a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da4da2da2da4540bec3d441d7cf8ca9, user_id=fd5f4301e59047acbfb758a1fd772d03, role_id=19f5e72c0be44a1197c7288de82e79e8)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:08.860942 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2aa44-ea65-44ea-80a1-b32f57e769a3 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:08.872039 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1150/2296] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:41:08 2022] PUT /identity/v3/projects/2da4da2da2da4540bec3d441d7cf8ca9/users/fd5f4301e59047acbfb758a1fd772d03/roles/19f5e72c0be44a1197c7288de82e79e8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:08.883594 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b38df-7683-4081-813c-ca1b50b572c0 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:08.883843 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b38df-7683-4081-813c-ca1b50b572c0 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:08.884120 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b38df-7683-4081-813c-ca1b50b572c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:08.911062 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-231b38df-7683-4081-813c-ca1b50b572c0 None None] MFA Rules not processed for user `fd5f4301e59047acbfb758a1fd772d03`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:08.912347 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-231b38df-7683-4081-813c-ca1b50b572c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:08.982811 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1147/2297] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:08 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:08.994884 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2aa44-ea65-44ea-80a1-b32f57e769a3 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:08.996670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-40d2aa44-ea65-44ea-80a1-b32f57e769a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:09.057348 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2aa44-ea65-44ea-80a1-b32f57e769a3 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:09.061823 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5812e9a-2f1f-4508-bc7d-e53750d42c90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:09.062598 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5812e9a-2f1f-4508-bc7d-e53750d42c90 service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:09.062765 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5812e9a-2f1f-4508-bc7d-e53750d42c90 service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:09.063037 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5812e9a-2f1f-4508-bc7d-e53750d42c90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:09.072774 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5812e9a-2f1f-4508-bc7d-e53750d42c90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:09.074253 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5812e9a-2f1f-4508-bc7d-e53750d42c90 service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:09.084041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1151/2298] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:41:08 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:41:09.135380 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b38df-7683-4081-813c-ca1b50b572c0 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:09.147355 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b38df-7683-4081-813c-ca1b50b572c0 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:09.151847 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8078012-e424-4dcf-af00-761f9739237c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:09.152535 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8078012-e424-4dcf-af00-761f9739237c service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:09.152818 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8078012-e424-4dcf-af00-761f9739237c service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:09.153115 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8078012-e424-4dcf-af00-761f9739237c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:09.214498 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8078012-e424-4dcf-af00-761f9739237c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:09.215936 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8078012-e424-4dcf-af00-761f9739237c service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:09.226471 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1148/2299] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:41:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:41:12.591732 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5812e9a-2f1f-4508-bc7d-e53750d42c90 service cinder] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:12.642175 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8078012-e424-4dcf-af00-761f9739237c service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:12.643610 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-a8078012-e424-4dcf-af00-761f9739237c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:12.667246 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5812e9a-2f1f-4508-bc7d-e53750d42c90 service cinder] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:12.672026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4129594-fd29-46e9-aee4-45dfbbf791b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:12.672821 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4129594-fd29-46e9-aee4-45dfbbf791b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:12.673123 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4129594-fd29-46e9-aee4-45dfbbf791b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:12.673406 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4129594-fd29-46e9-aee4-45dfbbf791b4 admin admin] PATH_INFO: `/v3/users/6ffaec80a3e94f498e1258a01dd986e9` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:12.674603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4129594-fd29-46e9-aee4-45dfbbf791b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ffaec80a3e94f498e1258a01dd986e9)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:12.676584 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4129594-fd29-46e9-aee4-45dfbbf791b4 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:12.690328 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=100 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:12.707872 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:12.718232 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8078012-e424-4dcf-af00-761f9739237c service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:12.725239 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223361ac-ef1b-480b-b1c6-c28e401c8587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:12.726209 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223361ac-ef1b-480b-b1c6-c28e401c8587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:12.726562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223361ac-ef1b-480b-b1c6-c28e401c8587 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:12.726919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223361ac-ef1b-480b-b1c6-c28e401c8587 admin admin] PATH_INFO: `/v3/users/c91e80d0ad7248cb88c4630925d20a00` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:12.728238 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223361ac-ef1b-480b-b1c6-c28e401c8587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c91e80d0ad7248cb88c4630925d20a00)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:12.730322 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223361ac-ef1b-480b-b1c6-c28e401c8587 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:12.733525 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-e4129594-fd29-46e9-aee4-45dfbbf791b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ffaec80a3e94f498e1258a01dd986e9'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:12.741774 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-e4129594-fd29-46e9-aee4-45dfbbf791b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ffaec80a3e94f498e1258a01dd986e9'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:12.742384 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-e4129594-fd29-46e9-aee4-45dfbbf791b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ffaec80a3e94f498e1258a01dd986e9'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:12.749747 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-e4129594-fd29-46e9-aee4-45dfbbf791b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ffaec80a3e94f498e1258a01dd986e9'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:12.750551 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-e4129594-fd29-46e9-aee4-45dfbbf791b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ffaec80a3e94f498e1258a01dd986e9'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:12.774820 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1152/2300] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:41:12 2022] DELETE /identity/v3/users/6ffaec80a3e94f498e1258a01dd986e9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:12.782807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431283f8-c2bf-4ac3-94b8-ad49df4a57d6 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:12.783341 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431283f8-c2bf-4ac3-94b8-ad49df4a57d6 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:12.783955 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431283f8-c2bf-4ac3-94b8-ad49df4a57d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:12.788275 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-223361ac-ef1b-480b-b1c6-c28e401c8587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c91e80d0ad7248cb88c4630925d20a00'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:12.796339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-223361ac-ef1b-480b-b1c6-c28e401c8587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c91e80d0ad7248cb88c4630925d20a00'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:12.797255 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-223361ac-ef1b-480b-b1c6-c28e401c8587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c91e80d0ad7248cb88c4630925d20a00'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:12.804138 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-431283f8-c2bf-4ac3-94b8-ad49df4a57d6 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:12.805469 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-431283f8-c2bf-4ac3-94b8-ad49df4a57d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:12.814046 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-223361ac-ef1b-480b-b1c6-c28e401c8587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c91e80d0ad7248cb88c4630925d20a00'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:12.824646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-223361ac-ef1b-480b-b1c6-c28e401c8587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c91e80d0ad7248cb88c4630925d20a00'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:12.831132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1149/2301] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:41:12 2022] DELETE /identity/v3/users/c91e80d0ad7248cb88c4630925d20a00 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:41:12.835570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dee1aa-6baf-4ac4-b04b-375979eeb393 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:12.836028 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dee1aa-6baf-4ac4-b04b-375979eeb393 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:12.836320 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dee1aa-6baf-4ac4-b04b-375979eeb393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:12.855766 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-d7dee1aa-6baf-4ac4-b04b-375979eeb393 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:12.856997 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d7dee1aa-6baf-4ac4-b04b-375979eeb393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:12.901643 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1153/2302] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:12.915787 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431283f8-c2bf-4ac3-94b8-ad49df4a57d6 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:12.917989 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-431283f8-c2bf-4ac3-94b8-ad49df4a57d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:12.934961 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1150/2303] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:41:12.945548 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dee1aa-6baf-4ac4-b04b-375979eeb393 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:12.953847 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-d7dee1aa-6baf-4ac4-b04b-375979eeb393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:12.991191 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431283f8-c2bf-4ac3-94b8-ad49df4a57d6 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:13.020677 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dee1aa-6baf-4ac4-b04b-375979eeb393 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:13.032448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74fca8-72d1-4323-9d29-5b24fd71da42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:13.032448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74fca8-72d1-4323-9d29-5b24fd71da42 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:13.032448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74fca8-72d1-4323-9d29-5b24fd71da42 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:13.032448 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74fca8-72d1-4323-9d29-5b24fd71da42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:13.035477 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df74fca8-72d1-4323-9d29-5b24fd71da42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:13.036929 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df74fca8-72d1-4323-9d29-5b24fd71da42 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:13.047697 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1151/2304] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:41:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:41:13.049159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c373a-28c2-4e3b-97e2-36c3bcfe808d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:13.049929 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c373a-28c2-4e3b-97e2-36c3bcfe808d service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:13.050218 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c373a-28c2-4e3b-97e2-36c3bcfe808d service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:13.050492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c373a-28c2-4e3b-97e2-36c3bcfe808d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:13.060943 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c373a-28c2-4e3b-97e2-36c3bcfe808d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:13.062423 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c373a-28c2-4e3b-97e2-36c3bcfe808d service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:13.072171 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1154/2305] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:41:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:41:13.437562 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74fca8-72d1-4323-9d29-5b24fd71da42 service neutron] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:13.439632 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-df74fca8-72d1-4323-9d29-5b24fd71da42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:13.453659 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:13.486791 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74fca8-72d1-4323-9d29-5b24fd71da42 service neutron] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:13.499868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c183a8-5861-4993-81b0-08110505afd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:13.501348 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c183a8-5861-4993-81b0-08110505afd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:13.502665 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c183a8-5861-4993-81b0-08110505afd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:13.506096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c183a8-5861-4993-81b0-08110505afd4 admin admin] PATH_INFO: `/v3/projects/f4734eaba7764c0a810861addab506b6` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:13.515462 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c183a8-5861-4993-81b0-08110505afd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4734eaba7764c0a810861addab506b6)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:13.518586 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c183a8-5861-4993-81b0-08110505afd4 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:13.573316 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c373a-28c2-4e3b-97e2-36c3bcfe808d service neutron] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:13.575092 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-5a8c373a-28c2-4e3b-97e2-36c3bcfe808d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:13.600160 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:13.613116 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f8c183a8-5861-4993-81b0-08110505afd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4734eaba7764c0a810861addab506b6', 'request_id': 'req-f8c183a8-5861-4993-81b0-08110505afd4'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:13.615281 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c373a-28c2-4e3b-97e2-36c3bcfe808d service neutron] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:13.626118 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f8c183a8-5861-4993-81b0-08110505afd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4734eaba7764c0a810861addab506b6', 'request_id': 'req-f8c183a8-5861-4993-81b0-08110505afd4'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:13.636092 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f8c183a8-5861-4993-81b0-08110505afd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4734eaba7764c0a810861addab506b6', 'request_id': 'req-f8c183a8-5861-4993-81b0-08110505afd4'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:13.648243 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f8c183a8-5861-4993-81b0-08110505afd4 admin admin] The token cache is being invalidate because project f4734eaba7764c0a810861addab506b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:41:13.648858 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f8c183a8-5861-4993-81b0-08110505afd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:13.649215 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-f8c183a8-5861-4993-81b0-08110505afd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:13.651938 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1152/2306] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:41:13 2022] DELETE /identity/v3/projects/f4734eaba7764c0a810861addab506b6 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:13.655755 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c183a8-5861-4993-81b0-08110505afd4 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:13.657149 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-f8c183a8-5861-4993-81b0-08110505afd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:13.690323 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd48a3-4d37-4194-91cc-759705d7b106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:13.691213 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd48a3-4d37-4194-91cc-759705d7b106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:13.691567 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd48a3-4d37-4194-91cc-759705d7b106 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:13.691915 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd48a3-4d37-4194-91cc-759705d7b106 admin admin] PATH_INFO: `/v3/projects/adb4d921f0fc4f1487a0bac99b59ebb3` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:13.693195 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd48a3-4d37-4194-91cc-759705d7b106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adb4d921f0fc4f1487a0bac99b59ebb3)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:13.695162 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd48a3-4d37-4194-91cc-759705d7b106 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:13.724923 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c183a8-5861-4993-81b0-08110505afd4 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:13.730217 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb371956-b909-4e48-9d62-9622e2c3ac7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:13.731085 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb371956-b909-4e48-9d62-9622e2c3ac7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:13.731361 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb371956-b909-4e48-9d62-9622e2c3ac7b admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:13.731640 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb371956-b909-4e48-9d62-9622e2c3ac7b admin admin] PATH_INFO: `/v3/users/33a5ec2921bc4569a8a183143132a350` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:13.732972 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb371956-b909-4e48-9d62-9622e2c3ac7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=33a5ec2921bc4569a8a183143132a350)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:13.735240 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb371956-b909-4e48-9d62-9622e2c3ac7b admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:13.747938 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-63cd48a3-4d37-4194-91cc-759705d7b106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adb4d921f0fc4f1487a0bac99b59ebb3', 'request_id': 'req-63cd48a3-4d37-4194-91cc-759705d7b106'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:13.756719 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-63cd48a3-4d37-4194-91cc-759705d7b106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adb4d921f0fc4f1487a0bac99b59ebb3', 'request_id': 'req-63cd48a3-4d37-4194-91cc-759705d7b106'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:13.764304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-63cd48a3-4d37-4194-91cc-759705d7b106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adb4d921f0fc4f1487a0bac99b59ebb3', 'request_id': 'req-63cd48a3-4d37-4194-91cc-759705d7b106'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:13.777241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-63cd48a3-4d37-4194-91cc-759705d7b106 admin admin] The token cache is being invalidate because project adb4d921f0fc4f1487a0bac99b59ebb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:41:13.777646 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-63cd48a3-4d37-4194-91cc-759705d7b106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:13.778209 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-63cd48a3-4d37-4194-91cc-759705d7b106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:13.780064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1155/2307] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:41:13 2022] DELETE /identity/v3/projects/adb4d921f0fc4f1487a0bac99b59ebb3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:13.783681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd48a3-4d37-4194-91cc-759705d7b106 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:13.785019 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-63cd48a3-4d37-4194-91cc-759705d7b106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:13.786660 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-fb371956-b909-4e48-9d62-9622e2c3ac7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33a5ec2921bc4569a8a183143132a350'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:13.795440 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-fb371956-b909-4e48-9d62-9622e2c3ac7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33a5ec2921bc4569a8a183143132a350'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:13.796323 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-fb371956-b909-4e48-9d62-9622e2c3ac7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33a5ec2921bc4569a8a183143132a350'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:13.809344 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-fb371956-b909-4e48-9d62-9622e2c3ac7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33a5ec2921bc4569a8a183143132a350'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:13.816335 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-fb371956-b909-4e48-9d62-9622e2c3ac7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33a5ec2921bc4569a8a183143132a350'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:13.823399 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1153/2308] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:41:13 2022] DELETE /identity/v3/users/33a5ec2921bc4569a8a183143132a350 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:41:13.852480 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd48a3-4d37-4194-91cc-759705d7b106 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:13.897871 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:13.917331 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eedbc8-91eb-4c4b-9b1b-a717fa470d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:13.918214 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eedbc8-91eb-4c4b-9b1b-a717fa470d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:13.918498 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eedbc8-91eb-4c4b-9b1b-a717fa470d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:13.918800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eedbc8-91eb-4c4b-9b1b-a717fa470d54 admin admin] PATH_INFO: `/v3/users/fd5f4301e59047acbfb758a1fd772d03` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:13.920126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eedbc8-91eb-4c4b-9b1b-a717fa470d54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd5f4301e59047acbfb758a1fd772d03)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:13.922141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eedbc8-91eb-4c4b-9b1b-a717fa470d54 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:13.987667 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-25eedbc8-91eb-4c4b-9b1b-a717fa470d54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd5f4301e59047acbfb758a1fd772d03'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.013235 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-25eedbc8-91eb-4c4b-9b1b-a717fa470d54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd5f4301e59047acbfb758a1fd772d03'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.013235 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-25eedbc8-91eb-4c4b-9b1b-a717fa470d54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd5f4301e59047acbfb758a1fd772d03'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.025150 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-25eedbc8-91eb-4c4b-9b1b-a717fa470d54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd5f4301e59047acbfb758a1fd772d03'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.026241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-25eedbc8-91eb-4c4b-9b1b-a717fa470d54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd5f4301e59047acbfb758a1fd772d03'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.052492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1156/2309] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:41:13 2022] DELETE /identity/v3/users/fd5f4301e59047acbfb758a1fd772d03 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:41:14.172248 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb371956-b909-4e48-9d62-9622e2c3ac7b admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:14.173835 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-fb371956-b909-4e48-9d62-9622e2c3ac7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:14.185672 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:14.190588 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb371956-b909-4e48-9d62-9622e2c3ac7b admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:14.279165 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e3c55-2359-47bb-af8f-b461e7de81a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:14.279612 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e3c55-2359-47bb-af8f-b461e7de81a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:14.279894 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e3c55-2359-47bb-af8f-b461e7de81a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:14.280169 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e3c55-2359-47bb-af8f-b461e7de81a6 admin admin] PATH_INFO: `/v3/projects/12a0c52727c249f7aa3191d620653394` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:14.281433 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e3c55-2359-47bb-af8f-b461e7de81a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12a0c52727c249f7aa3191d620653394)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:14.283387 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e3c55-2359-47bb-af8f-b461e7de81a6 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:14.340959 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4d0e3c55-2359-47bb-af8f-b461e7de81a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12a0c52727c249f7aa3191d620653394', 'request_id': 'req-4d0e3c55-2359-47bb-af8f-b461e7de81a6'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.348373 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4d0e3c55-2359-47bb-af8f-b461e7de81a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12a0c52727c249f7aa3191d620653394', 'request_id': 'req-4d0e3c55-2359-47bb-af8f-b461e7de81a6'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.354953 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4d0e3c55-2359-47bb-af8f-b461e7de81a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12a0c52727c249f7aa3191d620653394', 'request_id': 'req-4d0e3c55-2359-47bb-af8f-b461e7de81a6'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.366123 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4d0e3c55-2359-47bb-af8f-b461e7de81a6 admin admin] The token cache is being invalidate because project 12a0c52727c249f7aa3191d620653394 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:41:14.366415 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4d0e3c55-2359-47bb-af8f-b461e7de81a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.367188 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-4d0e3c55-2359-47bb-af8f-b461e7de81a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.369007 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1154/2310] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:41:14 2022] DELETE /identity/v3/projects/12a0c52727c249f7aa3191d620653394 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:14.373345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eedbc8-91eb-4c4b-9b1b-a717fa470d54 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:14.374876 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-25eedbc8-91eb-4c4b-9b1b-a717fa470d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:14.388659 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eedbc8-91eb-4c4b-9b1b-a717fa470d54 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:14.453447 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cb430-852a-480e-9d4d-feb020b56dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:14.454346 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cb430-852a-480e-9d4d-feb020b56dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:14.454606 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cb430-852a-480e-9d4d-feb020b56dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:14.454975 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cb430-852a-480e-9d4d-feb020b56dc7 admin admin] PATH_INFO: `/v3/users/b6252a52cd8a49c3accc8ecd1764a336` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:14.456207 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1cb430-852a-480e-9d4d-feb020b56dc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6252a52cd8a49c3accc8ecd1764a336)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:14.458246 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1cb430-852a-480e-9d4d-feb020b56dc7 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:14.463663 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e3c55-2359-47bb-af8f-b461e7de81a6 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:14.465286 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-4d0e3c55-2359-47bb-af8f-b461e7de81a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:14.480941 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e3c55-2359-47bb-af8f-b461e7de81a6 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:14.487151 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60189d99-dba4-4a7b-be60-7b25dc1743d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:14.488365 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60189d99-dba4-4a7b-be60-7b25dc1743d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:14.488727 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60189d99-dba4-4a7b-be60-7b25dc1743d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:14.489238 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60189d99-dba4-4a7b-be60-7b25dc1743d8 admin admin] PATH_INFO: `/v3/projects/2da4da2da2da4540bec3d441d7cf8ca9` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:14.490567 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60189d99-dba4-4a7b-be60-7b25dc1743d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2da4da2da2da4540bec3d441d7cf8ca9)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:14.492842 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60189d99-dba4-4a7b-be60-7b25dc1743d8 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:14.524051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1c1cb430-852a-480e-9d4d-feb020b56dc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6252a52cd8a49c3accc8ecd1764a336'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.531971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1c1cb430-852a-480e-9d4d-feb020b56dc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6252a52cd8a49c3accc8ecd1764a336'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.532680 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1c1cb430-852a-480e-9d4d-feb020b56dc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6252a52cd8a49c3accc8ecd1764a336'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.540043 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1c1cb430-852a-480e-9d4d-feb020b56dc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6252a52cd8a49c3accc8ecd1764a336'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.540916 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-1c1cb430-852a-480e-9d4d-feb020b56dc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6252a52cd8a49c3accc8ecd1764a336'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.542756 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:14.550720 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-60189d99-dba4-4a7b-be60-7b25dc1743d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2da4da2da2da4540bec3d441d7cf8ca9', 'request_id': 'req-60189d99-dba4-4a7b-be60-7b25dc1743d8'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.560360 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-60189d99-dba4-4a7b-be60-7b25dc1743d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2da4da2da2da4540bec3d441d7cf8ca9', 'request_id': 'req-60189d99-dba4-4a7b-be60-7b25dc1743d8'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.566233 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1157/2311] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:41:14 2022] DELETE /identity/v3/users/b6252a52cd8a49c3accc8ecd1764a336 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:14.566547 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-60189d99-dba4-4a7b-be60-7b25dc1743d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2da4da2da2da4540bec3d441d7cf8ca9', 'request_id': 'req-60189d99-dba4-4a7b-be60-7b25dc1743d8'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.579707 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-60189d99-dba4-4a7b-be60-7b25dc1743d8 admin admin] The token cache is being invalidate because project 2da4da2da2da4540bec3d441d7cf8ca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:41:14.579995 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-60189d99-dba4-4a7b-be60-7b25dc1743d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.580687 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-60189d99-dba4-4a7b-be60-7b25dc1743d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:14.582548 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1155/2312] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:41:14 2022] DELETE /identity/v3/projects/2da4da2da2da4540bec3d441d7cf8ca9 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:14.589294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac6a22-0836-41e8-99a7-ba70d9dceb01 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:14.589754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac6a22-0836-41e8-99a7-ba70d9dceb01 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:14.590033 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac6a22-0836-41e8-99a7-ba70d9dceb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:14.609678 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-99ac6a22-0836-41e8-99a7-ba70d9dceb01 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:14.610948 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-99ac6a22-0836-41e8-99a7-ba70d9dceb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:14.687797 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1158/2313] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:41:14.696192 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d233789-0390-4e17-83ef-e2160134756d None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:14.696721 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d233789-0390-4e17-83ef-e2160134756d None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:14.696721 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d233789-0390-4e17-83ef-e2160134756d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:14.722253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-4d233789-0390-4e17-83ef-e2160134756d None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:14.722253 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-4d233789-0390-4e17-83ef-e2160134756d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:14.743780 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1156/2314] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:41:14.748121 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac6a22-0836-41e8-99a7-ba70d9dceb01 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:14.758158 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac6a22-0836-41e8-99a7-ba70d9dceb01 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:14.762772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fe233-9978-4c83-98a9-89b380b61c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:14.763428 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fe233-9978-4c83-98a9-89b380b61c4b admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:14.763746 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fe233-9978-4c83-98a9-89b380b61c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:14.763977 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fe233-9978-4c83-98a9-89b380b61c4b admin admin] PATH_INFO: `/v3/domains` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:14.764696 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fe233-9978-4c83-98a9-89b380b61c4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:41:14.764998 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fe233-9978-4c83-98a9-89b380b61c4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:14.769617 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fe233-9978-4c83-98a9-89b380b61c4b admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:14.781773 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1159/2315] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:41:14.786115 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d233789-0390-4e17-83ef-e2160134756d None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:14.797766 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d233789-0390-4e17-83ef-e2160134756d None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:14.802661 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bac728-0393-4cf4-b1e6-3926d1542be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:14.803359 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bac728-0393-4cf4-b1e6-3926d1542be6 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:14.803760 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bac728-0393-4cf4-b1e6-3926d1542be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:14.803920 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bac728-0393-4cf4-b1e6-3926d1542be6 admin admin] PATH_INFO: `/v3/projects` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:14.804637 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bac728-0393-4cf4-b1e6-3926d1542be6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:14.806750 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bac728-0393-4cf4-b1e6-3926d1542be6 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:14.816577 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:14.841511 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1157/2316] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:41:14 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:41:14.843064 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fe233-9978-4c83-98a9-89b380b61c4b admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:14.854875 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fe233-9978-4c83-98a9-89b380b61c4b admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:14.872999 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bac728-0393-4cf4-b1e6-3926d1542be6 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:14.875438 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-a3bac728-0393-4cf4-b1e6-3926d1542be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:14.931937 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64bd22-656a-4508-9206-daed1c5d6090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:14.932733 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64bd22-656a-4508-9206-daed1c5d6090 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:14.933011 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64bd22-656a-4508-9206-daed1c5d6090 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:14.933331 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64bd22-656a-4508-9206-daed1c5d6090 admin admin] PATH_INFO: `/v3/users` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:14.934032 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64bd22-656a-4508-9206-daed1c5d6090 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:14.937045 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64bd22-656a-4508-9206-daed1c5d6090 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:14.950656 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bac728-0393-4cf4-b1e6-3926d1542be6 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:14.955479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fc728-f7f2-432d-801e-63ffd67d04ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:14.956801 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fc728-f7f2-432d-801e-63ffd67d04ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:14.956801 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fc728-f7f2-432d-801e-63ffd67d04ce admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:14.956987 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fc728-f7f2-432d-801e-63ffd67d04ce admin admin] PATH_INFO: `/v3/projects/24e862ba00e2495db17f66ca54849696` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:14.958195 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792fc728-f7f2-432d-801e-63ffd67d04ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=24e862ba00e2495db17f66ca54849696)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:14.960391 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792fc728-f7f2-432d-801e-63ffd67d04ce admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:14.964175 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1160/2317] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:41:14 2022] POST /identity/v3/users => generated 286 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:41:14.970592 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64bd22-656a-4508-9206-daed1c5d6090 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:14.981843 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64bd22-656a-4508-9206-daed1c5d6090 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:14.986629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca317f0-27a5-42d9-ae19-9d6247f5f3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:14.987287 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca317f0-27a5-42d9-ae19-9d6247f5f3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:14.987568 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca317f0-27a5-42d9-ae19-9d6247f5f3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:14.987839 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca317f0-27a5-42d9-ae19-9d6247f5f3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:14.988939 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca317f0-27a5-42d9-ae19-9d6247f5f3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:14.990303 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca317f0-27a5-42d9-ae19-9d6247f5f3f5 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:15.001921 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1161/2318] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:41:15.005778 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca317f0-27a5-42d9-ae19-9d6247f5f3f5 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:15.014251 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-792fc728-f7f2-432d-801e-63ffd67d04ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24e862ba00e2495db17f66ca54849696', 'request_id': 'req-792fc728-f7f2-432d-801e-63ffd67d04ce'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:15.016634 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca317f0-27a5-42d9-ae19-9d6247f5f3f5 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:15.024322 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-792fc728-f7f2-432d-801e-63ffd67d04ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24e862ba00e2495db17f66ca54849696', 'request_id': 'req-792fc728-f7f2-432d-801e-63ffd67d04ce'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:15.033387 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-792fc728-f7f2-432d-801e-63ffd67d04ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24e862ba00e2495db17f66ca54849696', 'request_id': 'req-792fc728-f7f2-432d-801e-63ffd67d04ce'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:15.047524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-792fc728-f7f2-432d-801e-63ffd67d04ce admin admin] The token cache is being invalidate because project 24e862ba00e2495db17f66ca54849696 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:41:15.047827 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-792fc728-f7f2-432d-801e-63ffd67d04ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:15.048513 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-792fc728-f7f2-432d-801e-63ffd67d04ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:15.057592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1158/2319] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:41:14 2022] DELETE /identity/v3/projects/24e862ba00e2495db17f66ca54849696 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:15.075511 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:15.105539 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c4546-bb59-4dd3-93d1-b9dae4d0ec56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:15.106301 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c4546-bb59-4dd3-93d1-b9dae4d0ec56 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:15.106647 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c4546-bb59-4dd3-93d1-b9dae4d0ec56 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:15.106925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c4546-bb59-4dd3-93d1-b9dae4d0ec56 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:15.108021 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c4546-bb59-4dd3-93d1-b9dae4d0ec56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:15.109909 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c4546-bb59-4dd3-93d1-b9dae4d0ec56 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:15.121203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1162/2320] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:41:15.125439 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fc728-f7f2-432d-801e-63ffd67d04ce admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:15.130360 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-792fc728-f7f2-432d-801e-63ffd67d04ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:15.146437 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fc728-f7f2-432d-801e-63ffd67d04ce admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:15.153268 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122097a-e8a2-4431-a3ee-c27b6cc9bb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:15.154483 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122097a-e8a2-4431-a3ee-c27b6cc9bb31 admin admin] REQUEST_METHOD: `PUT` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:15.154933 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122097a-e8a2-4431-a3ee-c27b6cc9bb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:15.155367 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122097a-e8a2-4431-a3ee-c27b6cc9bb31 admin admin] PATH_INFO: `/v3/projects/8d216ac735044816b784d0e5aeb23362/users/cf098fe9cc7e4c86aa66c27d48d3c831/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:15.173187 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122097a-e8a2-4431-a3ee-c27b6cc9bb31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d216ac735044816b784d0e5aeb23362, user_id=cf098fe9cc7e4c86aa66c27d48d3c831, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:15.175975 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122097a-e8a2-4431-a3ee-c27b6cc9bb31 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:15.191744 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1159/2321] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:41:15 2022] PUT /identity/v3/projects/8d216ac735044816b784d0e5aeb23362/users/cf098fe9cc7e4c86aa66c27d48d3c831/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:15.196858 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67352c41-cbd5-4808-a5ff-55575495bacc None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:15.197753 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67352c41-cbd5-4808-a5ff-55575495bacc None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:15.198036 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67352c41-cbd5-4808-a5ff-55575495bacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:15.216945 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-67352c41-cbd5-4808-a5ff-55575495bacc None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:15.218241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-67352c41-cbd5-4808-a5ff-55575495bacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:15.299331 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1163/2322] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:15.308042 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122097a-e8a2-4431-a3ee-c27b6cc9bb31 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:15.309807 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-0122097a-e8a2-4431-a3ee-c27b6cc9bb31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:15.373456 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122097a-e8a2-4431-a3ee-c27b6cc9bb31 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:15.377035 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9325d35-b085-4148-98f7-ed53c81ec06d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:15.377707 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9325d35-b085-4148-98f7-ed53c81ec06d service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:15.377986 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9325d35-b085-4148-98f7-ed53c81ec06d service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:15.378256 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9325d35-b085-4148-98f7-ed53c81ec06d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:15.388185 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9325d35-b085-4148-98f7-ed53c81ec06d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:15.389718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9325d35-b085-4148-98f7-ed53c81ec06d service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:15.393839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:15.402414 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1160/2323] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:41:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:41:16.067497 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50104360-a199-42b2-8fbe-5ad8a2ea946a None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:16.067986 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50104360-a199-42b2-8fbe-5ad8a2ea946a None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:16.068091 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50104360-a199-42b2-8fbe-5ad8a2ea946a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:16.100768 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-50104360-a199-42b2-8fbe-5ad8a2ea946a None None] MFA Rules not processed for user `cf098fe9cc7e4c86aa66c27d48d3c831`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:16.102059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-50104360-a199-42b2-8fbe-5ad8a2ea946a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:16.186917 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1164/2324] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:16.200907 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9325d35-b085-4148-98f7-ed53c81ec06d service neutron] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:16.202288 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-c9325d35-b085-4148-98f7-ed53c81ec06d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:16.272178 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9325d35-b085-4148-98f7-ed53c81ec06d service neutron] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:16.276346 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f88059-bf97-4fcf-9ff3-eb933e522055 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:16.277031 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f88059-bf97-4fcf-9ff3-eb933e522055 service cinder] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:16.277373 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f88059-bf97-4fcf-9ff3-eb933e522055 service cinder] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:16.277641 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f88059-bf97-4fcf-9ff3-eb933e522055 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:16.288592 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f88059-bf97-4fcf-9ff3-eb933e522055 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:16.290107 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f88059-bf97-4fcf-9ff3-eb933e522055 service cinder] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:16.300482 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1161/2325] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:41:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:41:18.734954 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbf9da-9137-400a-a4b3-6ddd9f7b50f1 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:18.735524 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbf9da-9137-400a-a4b3-6ddd9f7b50f1 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:18.735638 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbf9da-9137-400a-a4b3-6ddd9f7b50f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:18.752857 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-fdbbf9da-9137-400a-a4b3-6ddd9f7b50f1 None None] MFA Rules not processed for user `cf098fe9cc7e4c86aa66c27d48d3c831`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:18.754117 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-fdbbf9da-9137-400a-a4b3-6ddd9f7b50f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:18.761619 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1165/2326] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:24.901353 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f88059-bf97-4fcf-9ff3-eb933e522055 service cinder] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:24.903037 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-54f88059-bf97-4fcf-9ff3-eb933e522055 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:24.921614 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f88059-bf97-4fcf-9ff3-eb933e522055 service cinder] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:24.926317 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199581b3-11e1-43b5-96fe-690c1859bcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:24.927234 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199581b3-11e1-43b5-96fe-690c1859bcf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:24.927526 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199581b3-11e1-43b5-96fe-690c1859bcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:24.927805 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199581b3-11e1-43b5-96fe-690c1859bcf7 admin admin] PATH_INFO: `/v3/users/2d0cffe64ae7414bbc392675fbeed9de` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:24.929003 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199581b3-11e1-43b5-96fe-690c1859bcf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d0cffe64ae7414bbc392675fbeed9de)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:24.931207 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199581b3-11e1-43b5-96fe-690c1859bcf7 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:24.981040 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-199581b3-11e1-43b5-96fe-690c1859bcf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d0cffe64ae7414bbc392675fbeed9de'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:24.989719 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-199581b3-11e1-43b5-96fe-690c1859bcf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d0cffe64ae7414bbc392675fbeed9de'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:24.990341 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-199581b3-11e1-43b5-96fe-690c1859bcf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d0cffe64ae7414bbc392675fbeed9de'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:25.003029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-199581b3-11e1-43b5-96fe-690c1859bcf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d0cffe64ae7414bbc392675fbeed9de'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:25.010151 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-199581b3-11e1-43b5-96fe-690c1859bcf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d0cffe64ae7414bbc392675fbeed9de'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:25.016138 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1162/2327] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:41:24 2022] DELETE /identity/v3/users/2d0cffe64ae7414bbc392675fbeed9de => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:25.020622 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de8046-94fe-49bb-bf7f-5a938d8c0933 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:25.021183 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de8046-94fe-49bb-bf7f-5a938d8c0933 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:25.021313 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de8046-94fe-49bb-bf7f-5a938d8c0933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:25.040417 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-05de8046-94fe-49bb-bf7f-5a938d8c0933 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:25.042334 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-05de8046-94fe-49bb-bf7f-5a938d8c0933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:25.107577 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1166/2328] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:25.114783 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199581b3-11e1-43b5-96fe-690c1859bcf7 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:25.116237 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-199581b3-11e1-43b5-96fe-690c1859bcf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:25.179082 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199581b3-11e1-43b5-96fe-690c1859bcf7 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:25.182973 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721414e-e58a-4f58-8d2b-4033122e879a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:25.183775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721414e-e58a-4f58-8d2b-4033122e879a service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:25.184057 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721414e-e58a-4f58-8d2b-4033122e879a service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:25.184339 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721414e-e58a-4f58-8d2b-4033122e879a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:25.196049 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721414e-e58a-4f58-8d2b-4033122e879a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:25.197553 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721414e-e58a-4f58-8d2b-4033122e879a service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:25.208397 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1163/2329] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:41:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:41:25.604750 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de8046-94fe-49bb-bf7f-5a938d8c0933 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:25.606266 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-05de8046-94fe-49bb-bf7f-5a938d8c0933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:25.624554 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de8046-94fe-49bb-bf7f-5a938d8c0933 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:25.629528 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74b321-eb49-4697-bb9e-9ac684a5421a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:25.630359 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74b321-eb49-4697-bb9e-9ac684a5421a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:25.630653 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74b321-eb49-4697-bb9e-9ac684a5421a admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:25.634106 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74b321-eb49-4697-bb9e-9ac684a5421a admin admin] PATH_INFO: `/v3/projects/d9bb1d0f44ed46898a51f3d1be7e5268` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:25.635497 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df74b321-eb49-4697-bb9e-9ac684a5421a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9bb1d0f44ed46898a51f3d1be7e5268)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:25.637532 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df74b321-eb49-4697-bb9e-9ac684a5421a admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:25.639268 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:25.689256 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-df74b321-eb49-4697-bb9e-9ac684a5421a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9bb1d0f44ed46898a51f3d1be7e5268', 'request_id': 'req-df74b321-eb49-4697-bb9e-9ac684a5421a'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:25.702818 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-df74b321-eb49-4697-bb9e-9ac684a5421a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9bb1d0f44ed46898a51f3d1be7e5268', 'request_id': 'req-df74b321-eb49-4697-bb9e-9ac684a5421a'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:25.709629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-df74b321-eb49-4697-bb9e-9ac684a5421a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9bb1d0f44ed46898a51f3d1be7e5268', 'request_id': 'req-df74b321-eb49-4697-bb9e-9ac684a5421a'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:25.721513 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-df74b321-eb49-4697-bb9e-9ac684a5421a admin admin] The token cache is being invalidate because project d9bb1d0f44ed46898a51f3d1be7e5268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:41:25.721828 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-df74b321-eb49-4697-bb9e-9ac684a5421a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:25.722482 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-df74b321-eb49-4697-bb9e-9ac684a5421a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:25.724015 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1167/2330] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:41:25 2022] DELETE /identity/v3/projects/d9bb1d0f44ed46898a51f3d1be7e5268 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:35.203219 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=82,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:35.718160 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=24,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:39.731179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc896cd6-ba29-40b4-a7c0-941025c335ac None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:39.731811 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc896cd6-ba29-40b4-a7c0-941025c335ac None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:39.731811 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc896cd6-ba29-40b4-a7c0-941025c335ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:39.749851 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-dc896cd6-ba29-40b4-a7c0-941025c335ac None None] MFA Rules not processed for user `cf098fe9cc7e4c86aa66c27d48d3c831`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:39.751077 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-dc896cd6-ba29-40b4-a7c0-941025c335ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:39.812145 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1164/2331] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:41.820555 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48d895-d1ef-4ea1-98d9-3f26dbfe1fdd None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:41.821641 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48d895-d1ef-4ea1-98d9-3f26dbfe1fdd None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:41.821930 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48d895-d1ef-4ea1-98d9-3f26dbfe1fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:41.840395 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-2d48d895-d1ef-4ea1-98d9-3f26dbfe1fdd None None] MFA Rules not processed for user `654b47930e6f4a8ca378d6876cc8ae6c`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:41.841670 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-2d48d895-d1ef-4ea1-98d9-3f26dbfe1fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:41.903203 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1168/2332] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:49.803820 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=25 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:51.895325 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=25 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:55.779426 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc896cd6-ba29-40b4-a7c0-941025c335ac None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:55.781104 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-dc896cd6-ba29-40b4-a7c0-941025c335ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:55.852126 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc896cd6-ba29-40b4-a7c0-941025c335ac None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:55.857655 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46f9f3-7657-4276-a90d-8f57b96129ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:55.858549 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46f9f3-7657-4276-a90d-8f57b96129ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:55.858832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46f9f3-7657-4276-a90d-8f57b96129ef admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:55.859106 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46f9f3-7657-4276-a90d-8f57b96129ef admin admin] PATH_INFO: `/v3/users/bc8b20419fcf4269a24ff5b6f625d54d` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:55.860398 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46f9f3-7657-4276-a90d-8f57b96129ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc8b20419fcf4269a24ff5b6f625d54d)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:55.862382 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46f9f3-7657-4276-a90d-8f57b96129ef admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:55.909714 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-1f46f9f3-7657-4276-a90d-8f57b96129ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc8b20419fcf4269a24ff5b6f625d54d'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:55.918385 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-1f46f9f3-7657-4276-a90d-8f57b96129ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc8b20419fcf4269a24ff5b6f625d54d'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:55.919114 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-1f46f9f3-7657-4276-a90d-8f57b96129ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc8b20419fcf4269a24ff5b6f625d54d'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:55.931863 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-1f46f9f3-7657-4276-a90d-8f57b96129ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc8b20419fcf4269a24ff5b6f625d54d'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:55.939093 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-1f46f9f3-7657-4276-a90d-8f57b96129ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc8b20419fcf4269a24ff5b6f625d54d'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:55.945087 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1165/2333] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:41:55 2022] DELETE /identity/v3/users/bc8b20419fcf4269a24ff5b6f625d54d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:55.949779 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408a1ff-300b-47b2-b53a-cfd1acbd500f None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:55.950239 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408a1ff-300b-47b2-b53a-cfd1acbd500f None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:55.950356 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408a1ff-300b-47b2-b53a-cfd1acbd500f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:55.968749 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-4408a1ff-300b-47b2-b53a-cfd1acbd500f None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:55.970002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-4408a1ff-300b-47b2-b53a-cfd1acbd500f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:56.031528 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1169/2334] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:56.041278 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46f9f3-7657-4276-a90d-8f57b96129ef admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:56.042678 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-1f46f9f3-7657-4276-a90d-8f57b96129ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:56.102383 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46f9f3-7657-4276-a90d-8f57b96129ef admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:56.106460 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5dd4bd-4cb5-4690-a040-47ebad88ab05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:56.107118 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dd4bd-4cb5-4690-a040-47ebad88ab05 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:56.107398 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dd4bd-4cb5-4690-a040-47ebad88ab05 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:56.107678 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dd4bd-4cb5-4690-a040-47ebad88ab05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:56.118575 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5dd4bd-4cb5-4690-a040-47ebad88ab05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:56.120017 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5dd4bd-4cb5-4690-a040-47ebad88ab05 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:56.137539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1166/2335] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:41:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:41:56.404289 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408a1ff-300b-47b2-b53a-cfd1acbd500f None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:56.405809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-4408a1ff-300b-47b2-b53a-cfd1acbd500f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:56.417963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408a1ff-300b-47b2-b53a-cfd1acbd500f None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:56.423095 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f705e1-8a37-472d-b299-d1f6d0ae6123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:56.423966 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f705e1-8a37-472d-b299-d1f6d0ae6123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:56.424288 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f705e1-8a37-472d-b299-d1f6d0ae6123 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:56.424518 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f705e1-8a37-472d-b299-d1f6d0ae6123 admin admin] PATH_INFO: `/v3/projects/c59efd8a923f4b67b7fe8f6478cffe95` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:56.425771 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f705e1-8a37-472d-b299-d1f6d0ae6123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c59efd8a923f4b67b7fe8f6478cffe95)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:56.427698 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f705e1-8a37-472d-b299-d1f6d0ae6123 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:56.481425 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-20f705e1-8a37-472d-b299-d1f6d0ae6123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c59efd8a923f4b67b7fe8f6478cffe95', 'request_id': 'req-20f705e1-8a37-472d-b299-d1f6d0ae6123'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:56.490991 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-20f705e1-8a37-472d-b299-d1f6d0ae6123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c59efd8a923f4b67b7fe8f6478cffe95', 'request_id': 'req-20f705e1-8a37-472d-b299-d1f6d0ae6123'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:56.507562 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-20f705e1-8a37-472d-b299-d1f6d0ae6123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c59efd8a923f4b67b7fe8f6478cffe95', 'request_id': 'req-20f705e1-8a37-472d-b299-d1f6d0ae6123'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:56.519983 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-20f705e1-8a37-472d-b299-d1f6d0ae6123 admin admin] The token cache is being invalidate because project c59efd8a923f4b67b7fe8f6478cffe95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:41:56.520222 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-20f705e1-8a37-472d-b299-d1f6d0ae6123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:56.520893 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-20f705e1-8a37-472d-b299-d1f6d0ae6123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:41:56.525131 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1170/2336] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:41:56 2022] DELETE /identity/v3/projects/c59efd8a923f4b67b7fe8f6478cffe95 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:56.528161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f45144-3673-497e-9340-e6502b6b8f6b None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:56.528161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f45144-3673-497e-9340-e6502b6b8f6b None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:56.528919 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f45144-3673-497e-9340-e6502b6b8f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:56.563606 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-36f45144-3673-497e-9340-e6502b6b8f6b None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:56.565010 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-36f45144-3673-497e-9340-e6502b6b8f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:56.589662 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:56.677974 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1167/2337] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:56.687492 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd1591-39a9-453a-9f89-57149a20de4d None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:56.687787 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd1591-39a9-453a-9f89-57149a20de4d None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:56.688059 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd1591-39a9-453a-9f89-57149a20de4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:56.729116 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-3bcd1591-39a9-453a-9f89-57149a20de4d None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:56.730743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-3bcd1591-39a9-453a-9f89-57149a20de4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:56.754357 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1171/2338] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:56.755739 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f45144-3673-497e-9340-e6502b6b8f6b None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:56.772384 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f45144-3673-497e-9340-e6502b6b8f6b None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:56.782639 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4888c3-6f0a-4573-96e9-27d9ab82830a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:56.783320 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4888c3-6f0a-4573-96e9-27d9ab82830a admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:56.783798 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4888c3-6f0a-4573-96e9-27d9ab82830a admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:56.784047 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4888c3-6f0a-4573-96e9-27d9ab82830a admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:56.785547 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4888c3-6f0a-4573-96e9-27d9ab82830a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:41:56.786951 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4888c3-6f0a-4573-96e9-27d9ab82830a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:56.788636 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4888c3-6f0a-4573-96e9-27d9ab82830a admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:56.809795 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1168/2339] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:41:56.810881 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd1591-39a9-453a-9f89-57149a20de4d None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:56.822785 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd1591-39a9-453a-9f89-57149a20de4d None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:56.827653 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a4d17-839d-4522-8aa2-fe50efbc0400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:56.828327 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a4d17-839d-4522-8aa2-fe50efbc0400 admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:56.828618 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a4d17-839d-4522-8aa2-fe50efbc0400 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:56.828895 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a4d17-839d-4522-8aa2-fe50efbc0400 admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:56.829742 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a4d17-839d-4522-8aa2-fe50efbc0400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:56.831580 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a4d17-839d-4522-8aa2-fe50efbc0400 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:56.882342 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1172/2340] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:41:56 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:41:56.887146 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4888c3-6f0a-4573-96e9-27d9ab82830a admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:56.898400 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4888c3-6f0a-4573-96e9-27d9ab82830a admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:56.955781 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0dfce7-9ef3-4294-9234-48926fd58a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:56.956460 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0dfce7-9ef3-4294-9234-48926fd58a50 admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:56.956747 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0dfce7-9ef3-4294-9234-48926fd58a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:56.957023 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0dfce7-9ef3-4294-9234-48926fd58a50 admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:56.957765 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0dfce7-9ef3-4294-9234-48926fd58a50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:56.959690 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0dfce7-9ef3-4294-9234-48926fd58a50 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:56.982370 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1169/2341] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:41:56 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:41:56.986770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a4d17-839d-4522-8aa2-fe50efbc0400 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:56.997228 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a4d17-839d-4522-8aa2-fe50efbc0400 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:57.001802 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a216a6-f73d-4ec9-bdb1-9555cefd6ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:57.002455 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a216a6-f73d-4ec9-bdb1-9555cefd6ebf admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:57.002740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a216a6-f73d-4ec9-bdb1-9555cefd6ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:57.003018 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a216a6-f73d-4ec9-bdb1-9555cefd6ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:57.003754 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a216a6-f73d-4ec9-bdb1-9555cefd6ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:57.005391 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a216a6-f73d-4ec9-bdb1-9555cefd6ebf admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:57.016518 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1173/2342] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:41:57.020629 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0dfce7-9ef3-4294-9234-48926fd58a50 admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:57.031278 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0dfce7-9ef3-4294-9234-48926fd58a50 admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:57.036225 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510235ae-e062-40ac-9175-8025a60f0af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:57.043559 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510235ae-e062-40ac-9175-8025a60f0af1 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:57.043846 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510235ae-e062-40ac-9175-8025a60f0af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:57.044126 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510235ae-e062-40ac-9175-8025a60f0af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:57.044857 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510235ae-e062-40ac-9175-8025a60f0af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:57.046701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510235ae-e062-40ac-9175-8025a60f0af1 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:57.057683 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1170/2343] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:41:57.062005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a216a6-f73d-4ec9-bdb1-9555cefd6ebf admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:57.075108 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a216a6-f73d-4ec9-bdb1-9555cefd6ebf admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:57.079799 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42831205-0d96-43fc-9f68-a3dabd20f251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:57.080681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42831205-0d96-43fc-9f68-a3dabd20f251 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:57.080967 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42831205-0d96-43fc-9f68-a3dabd20f251 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:57.081285 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42831205-0d96-43fc-9f68-a3dabd20f251 admin admin] PATH_INFO: `/v3/projects/fd5d61e97d9743c2a79e0b751134ec4a/users/ea1fc13fa434452390d95f971a813a29/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:57.098181 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42831205-0d96-43fc-9f68-a3dabd20f251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd5d61e97d9743c2a79e0b751134ec4a, user_id=ea1fc13fa434452390d95f971a813a29, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:57.100867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42831205-0d96-43fc-9f68-a3dabd20f251 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:57.113539 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1174/2344] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:41:57 2022] PUT /identity/v3/projects/fd5d61e97d9743c2a79e0b751134ec4a/users/ea1fc13fa434452390d95f971a813a29/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:41:57.124151 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda49b8-2400-49d9-bd84-98e47ddbca35 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:57.124444 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda49b8-2400-49d9-bd84-98e47ddbca35 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:57.124729 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda49b8-2400-49d9-bd84-98e47ddbca35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:57.152956 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-efda49b8-2400-49d9-bd84-98e47ddbca35 None None] MFA Rules not processed for user `ea1fc13fa434452390d95f971a813a29`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:57.154267 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-efda49b8-2400-49d9-bd84-98e47ddbca35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:57.220156 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1171/2345] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:41:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:41:57.232329 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42831205-0d96-43fc-9f68-a3dabd20f251 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:57.234078 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-42831205-0d96-43fc-9f68-a3dabd20f251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:57.257248 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:57.295304 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42831205-0d96-43fc-9f68-a3dabd20f251 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:57.299034 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efc04e-7a5f-494f-96c8-5feed9ac97c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:57.299724 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efc04e-7a5f-494f-96c8-5feed9ac97c5 service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:57.300006 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efc04e-7a5f-494f-96c8-5feed9ac97c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:57.300281 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efc04e-7a5f-494f-96c8-5feed9ac97c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:57.309600 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efc04e-7a5f-494f-96c8-5feed9ac97c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:57.311045 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efc04e-7a5f-494f-96c8-5feed9ac97c5 service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:57.319928 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1175/2346] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:41:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:42:00.286136 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47333cdf-2726-455f-bdd4-03085290870a None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:00.286633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47333cdf-2726-455f-bdd4-03085290870a None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:00.286732 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47333cdf-2726-455f-bdd4-03085290870a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:00.305540 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-47333cdf-2726-455f-bdd4-03085290870a None None] MFA Rules not processed for user `b94e634e8855486a9ce14d3de9d986fb`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:00.307574 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-47333cdf-2726-455f-bdd4-03085290870a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:00.322572 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:00.385128 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1172/2347] 10.222.0.71 () {58 vars in 1009 bytes} [Thu Jul 21 20:42:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:42:00.440306 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efc04e-7a5f-494f-96c8-5feed9ac97c5 service cinder] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:00.441871 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-e3efc04e-7a5f-494f-96c8-5feed9ac97c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:00.536070 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efc04e-7a5f-494f-96c8-5feed9ac97c5 service cinder] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:00.542561 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838d82f-c2ca-4e36-a7bd-5db7466b2f40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3babb078203647e2b3ef3b2787b84e3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:00.543345 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838d82f-c2ca-4e36-a7bd-5db7466b2f40 service swift] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:00.543631 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838d82f-c2ca-4e36-a7bd-5db7466b2f40 service swift] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:00.543906 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838d82f-c2ca-4e36-a7bd-5db7466b2f40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:00.552889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838d82f-c2ca-4e36-a7bd-5db7466b2f40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:00.554709 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838d82f-c2ca-4e36-a7bd-5db7466b2f40 service swift] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:00.564071 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1176/2348] 10.222.0.71 () {60 vars in 1384 bytes} [Thu Jul 21 20:42:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 20:42:00.832736 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2dbf3a-7c37-49f2-92fe-9bfe00148339 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:00.833187 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2dbf3a-7c37-49f2-92fe-9bfe00148339 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:00.833567 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2dbf3a-7c37-49f2-92fe-9bfe00148339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:00.858284 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-fb2dbf3a-7c37-49f2-92fe-9bfe00148339 None None] MFA Rules not processed for user `ea1fc13fa434452390d95f971a813a29`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:00.860172 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-fb2dbf3a-7c37-49f2-92fe-9bfe00148339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:00.868333 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1173/2349] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:42:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:42:10.560674 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=49 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:10.847769 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=20 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:11.324832 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d01ae-33fe-4b06-b89c-9ca5710cfd87 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:11.325277 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d01ae-33fe-4b06-b89c-9ca5710cfd87 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:11.325657 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d01ae-33fe-4b06-b89c-9ca5710cfd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:11.345789 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-4e9d01ae-33fe-4b06-b89c-9ca5710cfd87 None None] MFA Rules not processed for user `b94e634e8855486a9ce14d3de9d986fb`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:11.347516 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-4e9d01ae-33fe-4b06-b89c-9ca5710cfd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:11.361216 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1177/2350] 10.222.0.71 () {58 vars in 1009 bytes} [Thu Jul 21 20:42:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:42:11.374039 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2dbf3a-7c37-49f2-92fe-9bfe00148339 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:11.388633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2dbf3a-7c37-49f2-92fe-9bfe00148339 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:11.394674 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7483d0-1f90-41c1-a531-be242d1a41f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3babb078203647e2b3ef3b2787b84e3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:11.396715 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7483d0-1f90-41c1-a531-be242d1a41f1 service swift] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:11.397151 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7483d0-1f90-41c1-a531-be242d1a41f1 service swift] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:11.397532 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7483d0-1f90-41c1-a531-be242d1a41f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:11.407915 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7483d0-1f90-41c1-a531-be242d1a41f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:11.410233 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7483d0-1f90-41c1-a531-be242d1a41f1 service swift] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:11.427360 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1174/2351] 10.222.0.71 () {60 vars in 1384 bytes} [Thu Jul 21 20:42:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 20:42:19.880462 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d01ae-33fe-4b06-b89c-9ca5710cfd87 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:19.882403 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-4e9d01ae-33fe-4b06-b89c-9ca5710cfd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:20.162241 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d01ae-33fe-4b06-b89c-9ca5710cfd87 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:20.172963 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d753e44-e726-4a9d-99f7-9ebfe29f9101 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '364e80ec79b94812a1df76c62c2b13a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:20.173918 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d753e44-e726-4a9d-99f7-9ebfe29f9101 service placement] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:20.174299 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d753e44-e726-4a9d-99f7-9ebfe29f9101 service placement] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:20.175366 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d753e44-e726-4a9d-99f7-9ebfe29f9101 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:20.178940 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-4d753e44-e726-4a9d-99f7-9ebfe29f9101 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:20.206337 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d753e44-e726-4a9d-99f7-9ebfe29f9101 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:20.209866 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d753e44-e726-4a9d-99f7-9ebfe29f9101 service placement] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:20.241106 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1178/2352] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:42:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:42:21.423982 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=6 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:22.602305 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0a3ab-f45f-4acc-807b-eaa7af0b2f68 None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:22.604359 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0a3ab-f45f-4acc-807b-eaa7af0b2f68 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:22.604892 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0a3ab-f45f-4acc-807b-eaa7af0b2f68 None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:22.606379 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1175/2353] 10.222.0.71 () {54 vars in 921 bytes} [Thu Jul 21 20:42:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:42:22.614540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0d60c-9842-4529-a2cd-7ed050ed0a56 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:22.614540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0d60c-9842-4529-a2cd-7ed050ed0a56 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:22.614540 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0d60c-9842-4529-a2cd-7ed050ed0a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:22.636526 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-eaa0d60c-9842-4529-a2cd-7ed050ed0a56 None None] MFA Rules not processed for user `6fbc5240f5214aaea4418052b94739e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:22.638123 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-eaa0d60c-9842-4529-a2cd-7ed050ed0a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:22.748683 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1179/2354] 10.222.0.71 () {58 vars in 1018 bytes} [Thu Jul 21 20:42:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:42:22.762330 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0a3ab-f45f-4acc-807b-eaa7af0b2f68 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:22.764606 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-a7a0a3ab-f45f-4acc-807b-eaa7af0b2f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:22.897335 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0a3ab-f45f-4acc-807b-eaa7af0b2f68 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:22.901691 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9c7aa-5b77-4680-8481-ac7359631fa6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2460e6a7e34a6881984e4f7ed294c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:22.902778 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9c7aa-5b77-4680-8481-ac7359631fa6 service glance] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:22.914097 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9c7aa-5b77-4680-8481-ac7359631fa6 service glance] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:22.914484 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9c7aa-5b77-4680-8481-ac7359631fa6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:22.942001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9c7aa-5b77-4680-8481-ac7359631fa6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:22.944889 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9c7aa-5b77-4680-8481-ac7359631fa6 service glance] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:22.987957 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1176/2355] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:42:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:42:23.107864 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0fde2-4432-4a3b-830c-40baf4f529f3 None None] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:23.108349 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0fde2-4432-4a3b-830c-40baf4f529f3 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:23.108729 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0fde2-4432-4a3b-830c-40baf4f529f3 None None] PATH_INFO: `/` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:23.110408 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1180/2356] 10.222.0.71 () {54 vars in 921 bytes} [Thu Jul 21 20:42:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:42:23.116776 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb56ea-cded-4399-8809-a16ccdbd4b50 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:23.117342 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb56ea-cded-4399-8809-a16ccdbd4b50 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:23.117832 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb56ea-cded-4399-8809-a16ccdbd4b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:23.148161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-79bb56ea-cded-4399-8809-a16ccdbd4b50 None None] MFA Rules not processed for user `03eee8ef92314296b7d5c14c52f0d080`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:23.149632 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-79bb56ea-cded-4399-8809-a16ccdbd4b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:23.230174 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1177/2357] 10.222.0.71 () {58 vars in 1018 bytes} [Thu Jul 21 20:42:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:42:23.242525 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0fde2-4432-4a3b-830c-40baf4f529f3 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:23.244918 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-efe0fde2-4432-4a3b-830c-40baf4f529f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:23.265347 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0fde2-4432-4a3b-830c-40baf4f529f3 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:23.269805 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc97649-1e52-43f9-b400-3f213c4266e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:23.272081 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc97649-1e52-43f9-b400-3f213c4266e2 service neutron] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:23.272603 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc97649-1e52-43f9-b400-3f213c4266e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:23.273003 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc97649-1e52-43f9-b400-3f213c4266e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:23.282800 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc97649-1e52-43f9-b400-3f213c4266e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:23.284510 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc97649-1e52-43f9-b400-3f213c4266e2 service neutron] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:23.296644 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1181/2358] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:42:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:42:23.428497 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c4259-ab89-4f97-82d7-0447a33407fa None None] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:23.428839 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c4259-ab89-4f97-82d7-0447a33407fa None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:23.429581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c4259-ab89-4f97-82d7-0447a33407fa None None] PATH_INFO: `/` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:23.432517 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1178/2359] 10.222.0.71 () {54 vars in 921 bytes} [Thu Jul 21 20:42:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:42:23.438865 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259925a-6882-494a-a3e8-1c11b7441302 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:23.439230 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259925a-6882-494a-a3e8-1c11b7441302 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:23.439576 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259925a-6882-494a-a3e8-1c11b7441302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:23.475200 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-d259925a-6882-494a-a3e8-1c11b7441302 None None] MFA Rules not processed for user `6fbc5240f5214aaea4418052b94739e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:23.476745 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-d259925a-6882-494a-a3e8-1c11b7441302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:23.492889 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1182/2360] 10.222.0.71 () {58 vars in 1018 bytes} [Thu Jul 21 20:42:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:42:23.504445 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c4259-ab89-4f97-82d7-0447a33407fa None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:23.517686 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c4259-ab89-4f97-82d7-0447a33407fa None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:23.523491 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0bfdb-43a9-444a-bf0f-68d4a52649a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:23.524156 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0bfdb-43a9-444a-bf0f-68d4a52649a1 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:23.524456 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0bfdb-43a9-444a-bf0f-68d4a52649a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:23.524768 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0bfdb-43a9-444a-bf0f-68d4a52649a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:23.539479 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f0bfdb-43a9-444a-bf0f-68d4a52649a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:23.541470 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f0bfdb-43a9-444a-bf0f-68d4a52649a1 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:23.554575 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1179/2361] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:42:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:42:26.753004 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf0dc9-0a4a-43e6-8319-014f8efa7235 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:26.754123 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf0dc9-0a4a-43e6-8319-014f8efa7235 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:26.754123 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf0dc9-0a4a-43e6-8319-014f8efa7235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:26.791086 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-d3cf0dc9-0a4a-43e6-8319-014f8efa7235 None None] MFA Rules not processed for user `24306a0e1b3c4ae08576a6b6762af0be`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:26.792630 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-d3cf0dc9-0a4a-43e6-8319-014f8efa7235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:26.901394 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:26.916467 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1183/2362] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:42:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:42:30.044430 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad139f-930e-4f1e-8a6d-cebf4a592858 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:30.045347 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad139f-930e-4f1e-8a6d-cebf4a592858 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:30.045531 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad139f-930e-4f1e-8a6d-cebf4a592858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:30.068925 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-5bad139f-930e-4f1e-8a6d-cebf4a592858 None None] MFA Rules not processed for user `b94e634e8855486a9ce14d3de9d986fb`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:30.070191 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-5bad139f-930e-4f1e-8a6d-cebf4a592858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:30.078892 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1180/2363] 10.222.0.71 () {58 vars in 967 bytes} [Thu Jul 21 20:42:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:42:30.093629 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf0dc9-0a4a-43e6-8319-014f8efa7235 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:30.106666 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf0dc9-0a4a-43e6-8319-014f8efa7235 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:30.112439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abd57f-1e7e-452c-9d19-e0c6dd886a2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3babb078203647e2b3ef3b2787b84e3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:30.113140 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abd57f-1e7e-452c-9d19-e0c6dd886a2f service swift] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:30.113425 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abd57f-1e7e-452c-9d19-e0c6dd886a2f service swift] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:30.113702 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abd57f-1e7e-452c-9d19-e0c6dd886a2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:30.123108 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76abd57f-1e7e-452c-9d19-e0c6dd886a2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:30.124717 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76abd57f-1e7e-452c-9d19-e0c6dd886a2f service swift] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:30.135818 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1184/2364] 10.222.0.71 () {60 vars in 1384 bytes} [Thu Jul 21 20:42:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 20:42:32.528113 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835693d-5181-4ce3-86ab-98e1bba9cd39 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:32.529374 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835693d-5181-4ce3-86ab-98e1bba9cd39 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:32.529880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835693d-5181-4ce3-86ab-98e1bba9cd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:32.551891 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-9835693d-5181-4ce3-86ab-98e1bba9cd39 None None] MFA Rules not processed for user `b94e634e8855486a9ce14d3de9d986fb`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:32.554469 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-9835693d-5181-4ce3-86ab-98e1bba9cd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:32.577621 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1181/2365] 10.222.0.71 () {58 vars in 1009 bytes} [Thu Jul 21 20:42:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:42:32.596262 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abd57f-1e7e-452c-9d19-e0c6dd886a2f service swift] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:32.621580 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abd57f-1e7e-452c-9d19-e0c6dd886a2f service swift] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:32.630537 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93c433-1023-4860-a12e-b7c3be273183 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3babb078203647e2b3ef3b2787b84e3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:32.632154 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93c433-1023-4860-a12e-b7c3be273183 service swift] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:32.632531 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93c433-1023-4860-a12e-b7c3be273183 service swift] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:32.632894 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93c433-1023-4860-a12e-b7c3be273183 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:32.663159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93c433-1023-4860-a12e-b7c3be273183 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:32.668173 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93c433-1023-4860-a12e-b7c3be273183 service swift] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:32.718199 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1185/2366] 10.222.0.71 () {60 vars in 1384 bytes} [Thu Jul 21 20:42:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 20:42:33.150450 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835693d-5181-4ce3-86ab-98e1bba9cd39 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:33.152570 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-9835693d-5181-4ce3-86ab-98e1bba9cd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:33.290073 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835693d-5181-4ce3-86ab-98e1bba9cd39 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:33.306522 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefadad4-8dd6-432e-8ef6-f71eab081c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:33.307688 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefadad4-8dd6-432e-8ef6-f71eab081c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:33.308162 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefadad4-8dd6-432e-8ef6-f71eab081c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:33.308601 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefadad4-8dd6-432e-8ef6-f71eab081c18 admin admin] PATH_INFO: `/v3/users/cf098fe9cc7e4c86aa66c27d48d3c831` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:33.310869 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefadad4-8dd6-432e-8ef6-f71eab081c18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf098fe9cc7e4c86aa66c27d48d3c831)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:33.313386 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefadad4-8dd6-432e-8ef6-f71eab081c18 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:33.376720 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:33.438775 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-cefadad4-8dd6-432e-8ef6-f71eab081c18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf098fe9cc7e4c86aa66c27d48d3c831'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:33.455502 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-cefadad4-8dd6-432e-8ef6-f71eab081c18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf098fe9cc7e4c86aa66c27d48d3c831'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:33.456071 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-cefadad4-8dd6-432e-8ef6-f71eab081c18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf098fe9cc7e4c86aa66c27d48d3c831'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:33.514322 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-cefadad4-8dd6-432e-8ef6-f71eab081c18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf098fe9cc7e4c86aa66c27d48d3c831'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:33.524668 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-cefadad4-8dd6-432e-8ef6-f71eab081c18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf098fe9cc7e4c86aa66c27d48d3c831'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:33.541125 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1182/2367] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:42:33 2022] DELETE /identity/v3/users/cf098fe9cc7e4c86aa66c27d48d3c831 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:42:33.825320 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93c433-1023-4860-a12e-b7c3be273183 service swift] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:33.838977 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93c433-1023-4860-a12e-b7c3be273183 service swift] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:34.010539 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bba88f-6d97-4189-9c15-4098974ec66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:34.011424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bba88f-6d97-4189-9c15-4098974ec66c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:34.011701 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bba88f-6d97-4189-9c15-4098974ec66c admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:34.011975 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bba88f-6d97-4189-9c15-4098974ec66c admin admin] PATH_INFO: `/v3/projects/8d216ac735044816b784d0e5aeb23362` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:34.019086 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bba88f-6d97-4189-9c15-4098974ec66c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d216ac735044816b784d0e5aeb23362)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:34.029061 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bba88f-6d97-4189-9c15-4098974ec66c admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:34.119190 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-e6bba88f-6d97-4189-9c15-4098974ec66c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d216ac735044816b784d0e5aeb23362', 'request_id': 'req-e6bba88f-6d97-4189-9c15-4098974ec66c'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:34.128950 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-e6bba88f-6d97-4189-9c15-4098974ec66c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d216ac735044816b784d0e5aeb23362', 'request_id': 'req-e6bba88f-6d97-4189-9c15-4098974ec66c'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:34.139519 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-e6bba88f-6d97-4189-9c15-4098974ec66c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d216ac735044816b784d0e5aeb23362', 'request_id': 'req-e6bba88f-6d97-4189-9c15-4098974ec66c'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:34.161190 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-e6bba88f-6d97-4189-9c15-4098974ec66c admin admin] The token cache is being invalidate because project 8d216ac735044816b784d0e5aeb23362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:42:34.161677 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-e6bba88f-6d97-4189-9c15-4098974ec66c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:34.162712 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-e6bba88f-6d97-4189-9c15-4098974ec66c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:34.166360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1186/2368] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:42:33 2022] DELETE /identity/v3/projects/8d216ac735044816b784d0e5aeb23362 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:42:34.173238 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450e3fa-dbe0-46d0-9996-f217322ac81e None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:34.173880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450e3fa-dbe0-46d0-9996-f217322ac81e None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:34.174373 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450e3fa-dbe0-46d0-9996-f217322ac81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:34.207631 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-8450e3fa-dbe0-46d0-9996-f217322ac81e None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:34.209018 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-8450e3fa-dbe0-46d0-9996-f217322ac81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:34.307624 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1183/2369] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:42:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:42:34.315985 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f9193-b01c-4a2a-99ee-d98f09f96faf None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:34.316295 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f9193-b01c-4a2a-99ee-d98f09f96faf None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:34.316555 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f9193-b01c-4a2a-99ee-d98f09f96faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:34.353803 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-bf7f9193-b01c-4a2a-99ee-d98f09f96faf None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:34.356091 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-bf7f9193-b01c-4a2a-99ee-d98f09f96faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:34.374294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1187/2370] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:42:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:42:34.379382 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450e3fa-dbe0-46d0-9996-f217322ac81e None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:34.393978 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450e3fa-dbe0-46d0-9996-f217322ac81e None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:34.399057 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db1a56-3ed9-468b-a280-b4f5187291ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:34.400086 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db1a56-3ed9-468b-a280-b4f5187291ec admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:34.400539 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db1a56-3ed9-468b-a280-b4f5187291ec admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:34.401924 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db1a56-3ed9-468b-a280-b4f5187291ec admin admin] PATH_INFO: `/v3/domains` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:34.403306 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db1a56-3ed9-468b-a280-b4f5187291ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:34.404034 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db1a56-3ed9-468b-a280-b4f5187291ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:34.406765 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db1a56-3ed9-468b-a280-b4f5187291ec admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:34.426840 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1184/2371] 10.222.0.71 () {58 vars in 1152 bytes} [Thu Jul 21 20:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:34.431681 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f9193-b01c-4a2a-99ee-d98f09f96faf None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:34.442763 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f9193-b01c-4a2a-99ee-d98f09f96faf None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:34.448575 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b1902-63eb-4a97-82ca-0e8e1d3b804c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:34.458420 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b1902-63eb-4a97-82ca-0e8e1d3b804c admin admin] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:34.458851 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b1902-63eb-4a97-82ca-0e8e1d3b804c admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:34.459141 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b1902-63eb-4a97-82ca-0e8e1d3b804c admin admin] PATH_INFO: `/v3/projects` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:34.460073 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b1902-63eb-4a97-82ca-0e8e1d3b804c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:34.462493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b1902-63eb-4a97-82ca-0e8e1d3b804c admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:34.506692 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1188/2372] 10.222.0.71 () {60 vars in 1152 bytes} [Thu Jul 21 20:42:34 2022] POST /identity/v3/projects => generated 367 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:42:34.511161 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db1a56-3ed9-468b-a280-b4f5187291ec admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:34.523096 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db1a56-3ed9-468b-a280-b4f5187291ec admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:34.600600 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829a51d-8c74-4ae0-8625-e8dd320cfa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:34.601724 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829a51d-8c74-4ae0-8625-e8dd320cfa7b admin admin] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:34.601875 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829a51d-8c74-4ae0-8625-e8dd320cfa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:34.602286 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829a51d-8c74-4ae0-8625-e8dd320cfa7b admin admin] PATH_INFO: `/v3/users` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:34.602841 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829a51d-8c74-4ae0-8625-e8dd320cfa7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:34.604882 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829a51d-8c74-4ae0-8625-e8dd320cfa7b admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:34.630220 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1185/2373] 10.222.0.71 () {60 vars in 1143 bytes} [Thu Jul 21 20:42:34 2022] POST /identity/v3/users => generated 284 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:42:34.646475 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b1902-63eb-4a97-82ca-0e8e1d3b804c admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:34.673002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b1902-63eb-4a97-82ca-0e8e1d3b804c admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:34.685961 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b63de-020e-4ffb-9549-7b82affedf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:34.686805 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b63de-020e-4ffb-9549-7b82affedf20 admin admin] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:34.687092 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b63de-020e-4ffb-9549-7b82affedf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:34.687370 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b63de-020e-4ffb-9549-7b82affedf20 admin admin] PATH_INFO: `/v3/roles` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:34.688114 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b63de-020e-4ffb-9549-7b82affedf20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:34.690051 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b63de-020e-4ffb-9549-7b82affedf20 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:34.752208 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1189/2374] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:42:34.757168 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829a51d-8c74-4ae0-8625-e8dd320cfa7b admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:34.768463 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829a51d-8c74-4ae0-8625-e8dd320cfa7b admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:34.774129 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5015f-36ae-4d9d-b3fd-7fd39293d473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:34.775017 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5015f-36ae-4d9d-b3fd-7fd39293d473 admin admin] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:34.775298 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5015f-36ae-4d9d-b3fd-7fd39293d473 admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:34.775568 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5015f-36ae-4d9d-b3fd-7fd39293d473 admin admin] PATH_INFO: `/v3/roles` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:34.776416 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5015f-36ae-4d9d-b3fd-7fd39293d473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:34.778608 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5015f-36ae-4d9d-b3fd-7fd39293d473 admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:34.793129 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1186/2375] 10.222.0.71 () {58 vars in 1121 bytes} [Thu Jul 21 20:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:42:34.797199 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b63de-020e-4ffb-9549-7b82affedf20 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:34.815522 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b63de-020e-4ffb-9549-7b82affedf20 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:34.822925 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cda4db-6fc6-4cc3-a656-4eed204d1027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:34.824017 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cda4db-6fc6-4cc3-a656-4eed204d1027 admin admin] REQUEST_METHOD: `PUT` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:34.824368 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cda4db-6fc6-4cc3-a656-4eed204d1027 admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:34.824809 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cda4db-6fc6-4cc3-a656-4eed204d1027 admin admin] PATH_INFO: `/v3/projects/b541e63fd43b45cba41e0e4b80e0fdc9/users/281a5adf7799458e92b03ae1e23bd41f/roles/8a0eeb2a21dc4897a58fb5851d58678c` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:34.843958 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cda4db-6fc6-4cc3-a656-4eed204d1027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b541e63fd43b45cba41e0e4b80e0fdc9, user_id=281a5adf7799458e92b03ae1e23bd41f, role_id=8a0eeb2a21dc4897a58fb5851d58678c)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:34.848030 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cda4db-6fc6-4cc3-a656-4eed204d1027 admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:34.858676 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:34.865005 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1190/2376] 10.222.0.71 () {60 vars in 1482 bytes} [Thu Jul 21 20:42:34 2022] PUT /identity/v3/projects/b541e63fd43b45cba41e0e4b80e0fdc9/users/281a5adf7799458e92b03ae1e23bd41f/roles/8a0eeb2a21dc4897a58fb5851d58678c => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:42:34.876001 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18617a06-a0c5-4b59-b3f2-4c2520856d15 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:34.885477 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18617a06-a0c5-4b59-b3f2-4c2520856d15 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:34.885477 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18617a06-a0c5-4b59-b3f2-4c2520856d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:34.900144 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:34.909996 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-18617a06-a0c5-4b59-b3f2-4c2520856d15 None None] MFA Rules not processed for user `281a5adf7799458e92b03ae1e23bd41f`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:34.913132 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-18617a06-a0c5-4b59-b3f2-4c2520856d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:35.003712 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1187/2377] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:42:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:42:35.023602 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cda4db-6fc6-4cc3-a656-4eed204d1027 admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:35.025495 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-21cda4db-6fc6-4cc3-a656-4eed204d1027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:35.123651 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cda4db-6fc6-4cc3-a656-4eed204d1027 admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:35.127578 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1b481-20b1-4a3e-9385-418b6bfcca88 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2460e6a7e34a6881984e4f7ed294c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:35.128247 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1b481-20b1-4a3e-9385-418b6bfcca88 service glance] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:35.128530 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1b481-20b1-4a3e-9385-418b6bfcca88 service glance] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:35.128807 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1b481-20b1-4a3e-9385-418b6bfcca88 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:35.166543 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1b481-20b1-4a3e-9385-418b6bfcca88 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:35.168012 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1b481-20b1-4a3e-9385-418b6bfcca88 service glance] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:35.203555 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1191/2378] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:42:35 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:42:43.894456 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ca1a1-c853-4d93-817a-299c2ff2eed3 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:43.895261 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ca1a1-c853-4d93-817a-299c2ff2eed3 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:43.895261 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ca1a1-c853-4d93-817a-299c2ff2eed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:43.923828 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-0a4ca1a1-c853-4d93-817a-299c2ff2eed3 None None] MFA Rules not processed for user `b94e634e8855486a9ce14d3de9d986fb`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:43.926285 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-0a4ca1a1-c853-4d93-817a-299c2ff2eed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:44.012103 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1188/2379] 10.222.0.71 () {58 vars in 1009 bytes} [Thu Jul 21 20:42:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:42:44.025505 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1b481-20b1-4a3e-9385-418b6bfcca88 service glance] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:44.027738 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a9e1b481-20b1-4a3e-9385-418b6bfcca88 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:44.129896 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1b481-20b1-4a3e-9385-418b6bfcca88 service glance] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:44.137927 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df73e114-da8d-48c7-9f4c-f52422c75ec9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3babb078203647e2b3ef3b2787b84e3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:44.138706 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73e114-da8d-48c7-9f4c-f52422c75ec9 service swift] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:44.139056 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73e114-da8d-48c7-9f4c-f52422c75ec9 service swift] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:44.139425 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73e114-da8d-48c7-9f4c-f52422c75ec9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:44.152450 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df73e114-da8d-48c7-9f4c-f52422c75ec9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:44.154513 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df73e114-da8d-48c7-9f4c-f52422c75ec9 service swift] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:44.165867 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1192/2380] 10.222.0.71 () {60 vars in 1384 bytes} [Thu Jul 21 20:42:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 20:42:44.862589 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5082f-481d-471b-8039-63fda30bfa5a None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:44.862880 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5082f-481d-471b-8039-63fda30bfa5a None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:44.863169 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5082f-481d-471b-8039-63fda30bfa5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:44.888786 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-8af5082f-481d-471b-8039-63fda30bfa5a None None] MFA Rules not processed for user `b94e634e8855486a9ce14d3de9d986fb`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:44.890251 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-8af5082f-481d-471b-8039-63fda30bfa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:44.901886 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1189/2381] 10.222.0.71 () {58 vars in 967 bytes} [Thu Jul 21 20:42:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:42:44.917849 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df73e114-da8d-48c7-9f4c-f52422c75ec9 service swift] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:44.935641 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df73e114-da8d-48c7-9f4c-f52422c75ec9 service swift] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:44.943202 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62405005-4b2f-498c-a7f5-06af1c853583 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3babb078203647e2b3ef3b2787b84e3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:44.944016 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62405005-4b2f-498c-a7f5-06af1c853583 service swift] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:44.944425 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62405005-4b2f-498c-a7f5-06af1c853583 service swift] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:44.944971 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62405005-4b2f-498c-a7f5-06af1c853583 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:44.963002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62405005-4b2f-498c-a7f5-06af1c853583 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:44.966334 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62405005-4b2f-498c-a7f5-06af1c853583 service swift] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:44.982041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1193/2382] 10.222.0.71 () {60 vars in 1384 bytes} [Thu Jul 21 20:42:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 20:42:45.279407 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194037ea-8079-41cd-86ef-2b90985df6bf None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:45.279890 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194037ea-8079-41cd-86ef-2b90985df6bf None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:45.280361 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194037ea-8079-41cd-86ef-2b90985df6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:45.312418 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-194037ea-8079-41cd-86ef-2b90985df6bf None None] MFA Rules not processed for user `b94e634e8855486a9ce14d3de9d986fb`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:45.314335 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-194037ea-8079-41cd-86ef-2b90985df6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:45.327023 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1190/2383] 10.222.0.71 () {58 vars in 1009 bytes} [Thu Jul 21 20:42:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:42:45.341939 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62405005-4b2f-498c-a7f5-06af1c853583 service swift] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:45.359097 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62405005-4b2f-498c-a7f5-06af1c853583 service swift] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:45.364795 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89349e3-2338-4b8b-adce-f71d183d23db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3babb078203647e2b3ef3b2787b84e3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:45.365485 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89349e3-2338-4b8b-adce-f71d183d23db service swift] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:45.365772 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89349e3-2338-4b8b-adce-f71d183d23db service swift] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:45.366041 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89349e3-2338-4b8b-adce-f71d183d23db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:45.376740 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89349e3-2338-4b8b-adce-f71d183d23db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:45.381045 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89349e3-2338-4b8b-adce-f71d183d23db service swift] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:45.394462 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1194/2384] 10.222.0.71 () {60 vars in 1384 bytes} [Thu Jul 21 20:42:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 20:42:45.809718 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f577b00-edd0-4d99-8f87-777f80910834 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:45.810005 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f577b00-edd0-4d99-8f87-777f80910834 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:45.810287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f577b00-edd0-4d99-8f87-777f80910834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:45.829633 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-2f577b00-edd0-4d99-8f87-777f80910834 None None] MFA Rules not processed for user `b94e634e8855486a9ce14d3de9d986fb`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:45.831035 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-2f577b00-edd0-4d99-8f87-777f80910834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:45.838640 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1191/2385] 10.222.0.71 () {58 vars in 967 bytes} [Thu Jul 21 20:42:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:42:45.858126 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89349e3-2338-4b8b-adce-f71d183d23db service swift] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:45.870884 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89349e3-2338-4b8b-adce-f71d183d23db service swift] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:45.876441 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb33c8a-6604-4228-8cda-5c1f6c9dfb74 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3babb078203647e2b3ef3b2787b84e3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:45.877130 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb33c8a-6604-4228-8cda-5c1f6c9dfb74 service swift] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:45.877566 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb33c8a-6604-4228-8cda-5c1f6c9dfb74 service swift] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:45.882078 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb33c8a-6604-4228-8cda-5c1f6c9dfb74 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:45.888612 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb33c8a-6604-4228-8cda-5c1f6c9dfb74 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:45.890491 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb33c8a-6604-4228-8cda-5c1f6c9dfb74 service swift] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:45.913438 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1195/2386] 10.222.0.71 () {60 vars in 1384 bytes} [Thu Jul 21 20:42:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 20:42:48.257836 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f577b00-edd0-4d99-8f87-777f80910834 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:48.259772 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-2f577b00-edd0-4d99-8f87-777f80910834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:48.341599 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f577b00-edd0-4d99-8f87-777f80910834 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:48.347351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670de2c-548f-4a7b-84ee-c0daa347edcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:48.348268 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670de2c-548f-4a7b-84ee-c0daa347edcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:48.348569 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670de2c-548f-4a7b-84ee-c0daa347edcc admin admin] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:48.348847 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670de2c-548f-4a7b-84ee-c0daa347edcc admin admin] PATH_INFO: `/v3/users/ea1fc13fa434452390d95f971a813a29` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:48.350145 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670de2c-548f-4a7b-84ee-c0daa347edcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea1fc13fa434452390d95f971a813a29)` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:48.352146 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670de2c-548f-4a7b-84ee-c0daa347edcc admin admin] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:48.386142 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:48.410293 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-0670de2c-548f-4a7b-84ee-c0daa347edcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea1fc13fa434452390d95f971a813a29'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:48.421309 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-0670de2c-548f-4a7b-84ee-c0daa347edcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea1fc13fa434452390d95f971a813a29'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:48.422056 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-0670de2c-548f-4a7b-84ee-c0daa347edcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea1fc13fa434452390d95f971a813a29'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:48.438069 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-0670de2c-548f-4a7b-84ee-c0daa347edcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea1fc13fa434452390d95f971a813a29'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:48.447507 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.notifications [None req-0670de2c-548f-4a7b-84ee-c0daa347edcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea1fc13fa434452390d95f971a813a29'} {{(pid=103664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:48.470847 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1192/2387] 10.222.0.71 () {58 vars in 1223 bytes} [Thu Jul 21 20:42:48 2022] DELETE /identity/v3/users/ea1fc13fa434452390d95f971a813a29 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:42:48.480493 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dd64f-bbe9-479e-aa3b-9e0cc3218557 None None] REQUEST_METHOD: `POST` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:48.481185 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dd64f-bbe9-479e-aa3b-9e0cc3218557 None None] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:48.481185 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dd64f-bbe9-479e-aa3b-9e0cc3218557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:48.506475 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.auth.core [None req-b26dd64f-bbe9-479e-aa3b-9e0cc3218557 None None] MFA Rules not processed for user `891d3014859343d8a558b44fef8a2f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:48.508424 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-b26dd64f-bbe9-479e-aa3b-9e0cc3218557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:48.590132 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=100 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:48.605838 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1196/2388] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:42:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:42:48.617029 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670de2c-548f-4a7b-84ee-c0daa347edcc admin admin] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:48.619083 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-0670de2c-548f-4a7b-84ee-c0daa347edcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:48.699025 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670de2c-548f-4a7b-84ee-c0daa347edcc admin admin] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:48.703682 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8194c79-c0a9-4ada-811e-8fed6a9bb8bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:48.704581 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8194c79-c0a9-4ada-811e-8fed6a9bb8bb service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:48.704868 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8194c79-c0a9-4ada-811e-8fed6a9bb8bb service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:48.705179 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8194c79-c0a9-4ada-811e-8fed6a9bb8bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:48.720273 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8194c79-c0a9-4ada-811e-8fed6a9bb8bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:48.723351 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8194c79-c0a9-4ada-811e-8fed6a9bb8bb service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:48.739627 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1193/2389] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:42:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:42:49.048447 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26dd64f-bbe9-479e-aa3b-9e0cc3218557 None None] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:49.051299 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-b26dd64f-bbe9-479e-aa3b-9e0cc3218557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:49.071159 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26dd64f-bbe9-479e-aa3b-9e0cc3218557 None None] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:49.077869 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02d7be-15bb-4494-a454-2730d4c74fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891d3014859343d8a558b44fef8a2f4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dd46499ce284e948f969ddbeb31196c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:49.079040 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02d7be-15bb-4494-a454-2730d4c74fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:49.079383 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02d7be-15bb-4494-a454-2730d4c74fee admin admin] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:49.079700 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02d7be-15bb-4494-a454-2730d4c74fee admin admin] PATH_INFO: `/v3/projects/fd5d61e97d9743c2a79e0b751134ec4a` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:49.081462 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02d7be-15bb-4494-a454-2730d4c74fee admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd5d61e97d9743c2a79e0b751134ec4a)` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:49.083618 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02d7be-15bb-4494-a454-2730d4c74fee admin admin] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:49.146709 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-4e02d7be-15bb-4494-a454-2730d4c74fee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd5d61e97d9743c2a79e0b751134ec4a', 'request_id': 'req-4e02d7be-15bb-4494-a454-2730d4c74fee'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:49.156770 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-4e02d7be-15bb-4494-a454-2730d4c74fee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd5d61e97d9743c2a79e0b751134ec4a', 'request_id': 'req-4e02d7be-15bb-4494-a454-2730d4c74fee'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:49.164382 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-4e02d7be-15bb-4494-a454-2730d4c74fee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd5d61e97d9743c2a79e0b751134ec4a', 'request_id': 'req-4e02d7be-15bb-4494-a454-2730d4c74fee'} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:49.177613 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-4e02d7be-15bb-4494-a454-2730d4c74fee admin admin] The token cache is being invalidate because project fd5d61e97d9743c2a79e0b751134ec4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:42:49.177918 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-4e02d7be-15bb-4494-a454-2730d4c74fee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:49.178595 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.notifications [None req-4e02d7be-15bb-4494-a454-2730d4c74fee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:42:49.180200 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1197/2390] 10.222.0.71 () {58 vars in 1232 bytes} [Thu Jul 21 20:42:49 2022] DELETE /identity/v3/projects/fd5d61e97d9743c2a79e0b751134ec4a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:42:57.828281 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49eea1-414c-41e5-8356-de06b3b0af36 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:57.828860 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49eea1-414c-41e5-8356-de06b3b0af36 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:57.828967 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49eea1-414c-41e5-8356-de06b3b0af36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:57.849015 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-dd49eea1-414c-41e5-8356-de06b3b0af36 None None] MFA Rules not processed for user `b94e634e8855486a9ce14d3de9d986fb`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:57.850280 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-dd49eea1-414c-41e5-8356-de06b3b0af36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:57.910535 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1194/2391] 10.222.0.71 () {58 vars in 967 bytes} [Thu Jul 21 20:42:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:42:57.923002 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02d7be-15bb-4494-a454-2730d4c74fee admin admin] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:57.924434 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-4e02d7be-15bb-4494-a454-2730d4c74fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:57.999360 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02d7be-15bb-4494-a454-2730d4c74fee admin admin] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:58.004402 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731de298-660c-4005-9a7d-92161f8fc3d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3babb078203647e2b3ef3b2787b84e3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:58.005088 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731de298-660c-4005-9a7d-92161f8fc3d4 service swift] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:58.005382 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731de298-660c-4005-9a7d-92161f8fc3d4 service swift] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:58.005664 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731de298-660c-4005-9a7d-92161f8fc3d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:58.015224 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731de298-660c-4005-9a7d-92161f8fc3d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:58.016922 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731de298-660c-4005-9a7d-92161f8fc3d4 service swift] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:58.025865 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1198/2392] 10.222.0.71 () {60 vars in 1384 bytes} [Thu Jul 21 20:42:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 20:43:07.903646 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=70,keystone:DELETE=5,keystone:INSERT=1 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:43:08.022290 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=62,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:15.701915 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49eea1-414c-41e5-8356-de06b3b0af36 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:15.703429 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-dd49eea1-414c-41e5-8356-de06b3b0af36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:15.759854 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49eea1-414c-41e5-8356-de06b3b0af36 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:15.763331 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f54b7d-3c19-4119-a291-7b7ec98c0415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:15.763969 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f54b7d-3c19-4119-a291-7b7ec98c0415 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:15.764251 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f54b7d-3c19-4119-a291-7b7ec98c0415 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:15.764524 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f54b7d-3c19-4119-a291-7b7ec98c0415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:15.766325 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-10f54b7d-3c19-4119-a291-7b7ec98c0415 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:15.836434 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f54b7d-3c19-4119-a291-7b7ec98c0415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:15.837911 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f54b7d-3c19-4119-a291-7b7ec98c0415 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:15.849948 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1195/2393] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:20.079517 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731de298-660c-4005-9a7d-92161f8fc3d4 service swift] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:20.080874 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-731de298-660c-4005-9a7d-92161f8fc3d4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:20.096914 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731de298-660c-4005-9a7d-92161f8fc3d4 service swift] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:20.103105 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b581c1-6361-4829-bc19-f75a67e09f45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fbc5240f5214aaea4418052b94739e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:20.103743 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b581c1-6361-4829-bc19-f75a67e09f45 service nova] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:20.104026 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b581c1-6361-4829-bc19-f75a67e09f45 service nova] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:20.104302 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b581c1-6361-4829-bc19-f75a67e09f45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:20.106795 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-77b581c1-6361-4829-bc19-f75a67e09f45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:20.167817 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b581c1-6361-4829-bc19-f75a67e09f45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:20.170390 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b581c1-6361-4829-bc19-f75a67e09f45 service nova] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:20.187414 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1199/2394] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:22.998918 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750d655-cca7-4c4b-9177-151759c26866 None None] REQUEST_METHOD: `POST` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:22.999484 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750d655-cca7-4c4b-9177-151759c26866 None None] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:22.999484 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750d655-cca7-4c4b-9177-151759c26866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:23.016223 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.auth.core [None req-e750d655-cca7-4c4b-9177-151759c26866 None None] MFA Rules not processed for user `654b47930e6f4a8ca378d6876cc8ae6c`. Rule list: `[]` (Enabled: `True`). {{(pid=103664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:23.017493 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-e750d655-cca7-4c4b-9177-151759c26866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:23.072019 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1196/2395] 10.222.0.71 () {58 vars in 957 bytes} [Thu Jul 21 20:44:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:23.171350 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b581c1-6361-4829-bc19-f75a67e09f45 service nova] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:23.172846 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-77b581c1-6361-4829-bc19-f75a67e09f45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:23.228939 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b581c1-6361-4829-bc19-f75a67e09f45 service nova] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:23.233278 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201faea-e8a7-4ee9-bee7-84b9005e73fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:23.233917 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201faea-e8a7-4ee9-bee7-84b9005e73fe service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:23.234205 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201faea-e8a7-4ee9-bee7-84b9005e73fe service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:23.234481 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201faea-e8a7-4ee9-bee7-84b9005e73fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:23.236291 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-d201faea-e8a7-4ee9-bee7-84b9005e73fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:23.249831 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201faea-e8a7-4ee9-bee7-84b9005e73fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:23.251264 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201faea-e8a7-4ee9-bee7-84b9005e73fe service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:23.262293 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1200/2396] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:26.968764 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750d655-cca7-4c4b-9177-151759c26866 None None] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:26.979319 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750d655-cca7-4c4b-9177-151759c26866 None None] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:26.983541 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba8766-92fc-48dd-bb18-24f93a3b1d3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:26.984214 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba8766-92fc-48dd-bb18-24f93a3b1d3f service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:26.984497 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba8766-92fc-48dd-bb18-24f93a3b1d3f service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:26.984770 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba8766-92fc-48dd-bb18-24f93a3b1d3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:26.987793 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-7dba8766-92fc-48dd-bb18-24f93a3b1d3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:26.998930 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba8766-92fc-48dd-bb18-24f93a3b1d3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:27.000296 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba8766-92fc-48dd-bb18-24f93a3b1d3f service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:27.009587 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1197/2397] 10.222.0.71 () {60 vars in 1396 bytes} [Thu Jul 21 20:44:26 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:27.118294 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201faea-e8a7-4ee9-bee7-84b9005e73fe service cinder] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:27.127063 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201faea-e8a7-4ee9-bee7-84b9005e73fe service cinder] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:27.130835 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e92a64-7409-44b6-ab08-43553d0fa692 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34db429cd31549f78daf5b1cbf6f0bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:27.131482 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e92a64-7409-44b6-ab08-43553d0fa692 service cinder] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:27.131806 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e92a64-7409-44b6-ab08-43553d0fa692 service cinder] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:27.132042 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e92a64-7409-44b6-ab08-43553d0fa692 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:27.133840 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a0e92a64-7409-44b6-ab08-43553d0fa692 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:27.196832 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e92a64-7409-44b6-ab08-43553d0fa692 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:27.198317 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e92a64-7409-44b6-ab08-43553d0fa692 service cinder] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:27.207918 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1201/2398] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:37.005287 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=85 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:37.203735 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=82 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:04.820622 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba8766-92fc-48dd-bb18-24f93a3b1d3f service neutron] Authenticating user token {{(pid=103664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:04.822007 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-7dba8766-92fc-48dd-bb18-24f93a3b1d3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:04.832816 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba8766-92fc-48dd-bb18-24f93a3b1d3f service neutron] Validating token access rules against request {{(pid=103664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:04.836462 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea85a8-c646-4458-bfd0-6c4003caa777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03eee8ef92314296b7d5c14c52f0d080', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:04.837093 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea85a8-c646-4458-bfd0-6c4003caa777 service neutron] REQUEST_METHOD: `GET` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:04.837396 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea85a8-c646-4458-bfd0-6c4003caa777 service neutron] SCRIPT_NAME: `/identity` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:04.837680 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea85a8-c646-4458-bfd0-6c4003caa777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:04.839397 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.fernet_utils [None req-3aea85a8-c646-4458-bfd0-6c4003caa777 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:04.849030 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea85a8-c646-4458-bfd0-6c4003caa777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:04.850523 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea85a8-c646-4458-bfd0-6c4003caa777 service neutron] RBAC: Authorization granted {{(pid=103664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:04.859701 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: [pid: 103664|app: 0|req: 1198/2399] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:45:14.856239 dv-cinder-820027-13-iscsi devstack@keystone.service[103664]: DEBUG dbcounter [-] [103664] Writing DB stats keystone:SELECT=6 {{(pid=103664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:47:20.565230 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e92a64-7409-44b6-ab08-43553d0fa692 service cinder] Authenticating user token {{(pid=103663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:47:20.567208 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-a0e92a64-7409-44b6-ab08-43553d0fa692 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:47:20.706466 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e92a64-7409-44b6-ab08-43553d0fa692 service cinder] Validating token access rules against request {{(pid=103663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:47:20.712472 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db76ab-e3b6-45a1-a5f3-5aee30e7ea07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '364e80ec79b94812a1df76c62c2b13a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37e4afa595894a9994a8e6981ebb7c41', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:47:20.713153 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db76ab-e3b6-45a1-a5f3-5aee30e7ea07 service placement] REQUEST_METHOD: `GET` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:47:20.713439 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db76ab-e3b6-45a1-a5f3-5aee30e7ea07 service placement] SCRIPT_NAME: `/identity` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:47:20.713720 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db76ab-e3b6-45a1-a5f3-5aee30e7ea07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:47:20.715755 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.fernet_utils [None req-24db76ab-e3b6-45a1-a5f3-5aee30e7ea07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:47:20.727979 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db76ab-e3b6-45a1-a5f3-5aee30e7ea07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:47:20.730352 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db76ab-e3b6-45a1-a5f3-5aee30e7ea07 service placement] RBAC: Authorization granted {{(pid=103663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:47:20.742427 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: [pid: 103663|app: 0|req: 1202/2400] 10.222.0.71 () {60 vars in 1365 bytes} [Thu Jul 21 20:47:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:47:30.737387 dv-cinder-820027-13-iscsi devstack@keystone.service[103663]: DEBUG dbcounter [-] [103663] Writing DB stats keystone:SELECT=63 {{(pid=103663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}